Google Launched 'MusicLM', Its Answer To OpenAI's ChatGPT, But For Music

Robot hand piano

Google is stepping up its game in the AI field, and it's doing it with purpose.

In the past, the company has been extensively using AI to power its products. But rarely did it use its AI products to allow others create products on top of it. Since OpenAI debuted ChatGPT, the company sent a tremor throughout the AI ecosystem, that it struck fear deep into Google's heart.

Even Sundar Pichai issued a "code red".

The CEO has summoned both Google founders Larry Page and Sergey Brin for help, and that the company is now taking its approach in a more direct manner.

This time, at one of its tries, the company is aiming at the music industry.

Google said that it has published a research on what it calls the 'MusicLM', which is an AI-powered system capable of creating music in any genre with only text description.

While MusicLM isn't the first of its kind, because in the past, projects like Google's AudioML and OpenAI's Jukebox exist, but MusicLM is developed differently.

This is because the AI model has been trained with a massive database worth more than 280,000 hours of music.

An electric guitar:

A professional piano player:

An opera singer:

This in turn should help it compose music with surprising variety and depth.

Not only that the AI can combine different genres and instruments together to create a masterpiece, the AI can also write tracks using abstract concepts that are normally difficult for computers to understand.

So if users want a hybrid of dance music and reggaeton with a "spacey, otherworldly" tune that evokes a "sense of wonder and awe," MusicLM can create that.

The AI can even tune melodies based on humming, whistling or the description of a painting.

Users can also use its story mode feature, which can stitch several descriptions together to produce a DJ set or soundtrack.

A combination of jazz, pop, rock, death metal, rap, string quartet with violins, epic movie soundtrack with drums, and Scottish folk song with traditional instruments, into one track:

Additionally, MusicLM can also be directed by a combination of a picture and a caption, and that it can produce music that is "played" by a certain kind of instrument in a particular style.

For all it's worth, MusicLM is like the ChatGPT for music.

The 'Scream' painting by Edvard Munch, with the description: "Inspired by a hallucinatory experience in which Munch felt and heard a scream throughout nature, it depicts a panic-stricken creature, simultaneously corpse like and reminiscent of a sperm or fetus, whose contours are echoed in the swirling lines of the blood-red sky":

It isn't the first generative AI system for music, but it is certainly the first with the ability to create songs with "high-fidelity" and "substantial complexity."

While the technology promises many things, it comes with several downsides. MusicLM has its problems, as with many AI generators.

First, some compositions sound strange, and vocals tend to be incomprehensible.

While the performances themselves are better than what most people would expect, results can be repetitive in ways human composers won't ever make. And while the system can technically synthesize vocals, the results are still not ideal and have problems like distorted samples.

Second, the AI generator from Google was trained using copyrighted materials. What this means, there are chances that the songs produced by the AI can contain one or several copyrighted materials-influenced tunes.

While questions regarding licensing for AI music haven't been settled, researches and whitepapers suggest that there's enough "coherent" traces of the original sounds that AI music can violate reproduction rights. While people may have to get clearances to release AI-created songs, things may not be that easy.

Due to this concerns, this is why the researchers behind MusicLM aren't releasing the AI to the public.

"We strongly emphasize the need for more future work in tackling these risks associated to music generation — we have no plans to release models at this point," the study states.

At this time, the time when Google announced the AI, about 1% of the music produced was copied directly from the training songs.