AIs can work in ways that many people find them confusing. But nevertheless, the technology have proven itself to be quite extraordinary in many cases.
Some of which, was when researchers put AI to the test in the fields of art. In many researches, AIs can be made to generate faces of people that don't exist, artistic nude arts, and more.
Then there are instances where AIs were trained to create heavy metal songs, capable of growling death metal, playing plausible-sounding Jazz music, and more.
Researchers found that AI in its ability to create artworks is because creativity is somehow not limited to just humans.
And here, the people at OpenAI has created 'Jukebox'.
This AI is able to create entire songs complete with music, lyrics, and vocals.
According to OpenAI on its blog post:
The open-sourced AI was trained on massive data sets of music across nearly every genre.
"To train this model, we crawled the web to curate a new dataset of 1.2 million songs (600,000 of which are in English), paired with the corresponding lyrics and metadata from LyricWiki," said OpenAI.
"The metadata includes artist, album genre, and year of the songs, along with common moods or playlist keywords associated with each song. We train on 32-bit, 44.1 kHz raw audio, and perform data augmentation by randomly downmixing the right and left channels to produce mono audio."
Then, the researchers tweaked the top-level transformer to train it on predicting compressed audio tokens. This way, the researchers can provide additional information, such as the artist and genre for each song
As a result, the AI can create completely new songs that, in many cases, sound very much like the artist(s) it was trained on.
Related: The 'GPT-2' AI Could Create Significant Societal Impacts OpenAI Is Scared Of
Much like this AI that learned how to transfer the style of an artwork from one image to another, Jukebox is able to figure out how to imitate the style and genre of music, and creates a fresh new song.
It can even attempt to recreate the lyrical and singing style of specific singers.
As explained by the OpenAI team’s research paper:
"We can also generate novel completions of existing songs. Our approach allows the option to influence the generation process: by swapping the top prior with a conditional prior, we can condition on lyrics to tell the singer what to sing, or on midi to control the composition."
Initially, the applications for Jukebox are mostly academic.
People also can’t get Jukebox to generate new music just yet, and it takes about nine hours for the AI to render a single minute of music. This essentially puts any large-scale interactive uses out of reach for the time being.
While Jukebox represents a step forward in musical quality, coherence, length of audio sample, and ability to condition on artist, genre, and lyrics, there is a significant limitation to what it can and cannot do.
For example, while the generated songs show local musical coherence, follow traditional chord patterns, and can even feature impressive solos, the researchers said that they do not hear familiar larger musical structures such as choruses that repeat.
"Our downsampling and upsampling process introduces discernable noise. Improving the VQ-VAE so its codes capture more musical information would help reduce this. Our models are also slow to sample from, because of the autoregressive nature of sampling."
In the meantime, OpenAI has given some samples of the AI's work: