2020 is almost over, and 'COVID-19' coronavirus is still here. With the holiday coming, Google has something in its mind to festive.
And that is by introducing four singing blobs called 'Blob Opera'. Developed by Google and AI artist David Li, it's essentially a machine learning experiment that allows users compose their own operatic renditions through a quartet of colorful singing blobs.
To make use of these blobs, users can simply drag the blobs up and down to change their pitch, while moving the blobs side-to-side to produce a range of vowel sounds.
And just for the holiday, users can make the blobs recreate Christmas classics such as The First Noel or Joy to the World.
It looks like a simple web application, boasting animations and some rendered voices.
But behind the scene, Blob Opera wasn't easy to build.
The AI that powers Blob Opera was trained with 16 hours-worth of singing from four professionals (Cristian Joel [tenor], Frederick Tong [bass], Joanna Gamble [mezzo‑soprano], and Olivia Doutney [soprano]) By learning from the professionals, Blob Opera uses its AI to synthesize the noises it produces.
“In the experiment, you don’t hear their voices, but the machine learning model’s understanding of what opera singing sounds like, based on what it learnt from them,” said Google on a blog post..
Long story short, the AI is smart. But Blob Opera isn't at all revolutionary. It doesn't introduce anything much to the table.
In the past, similar AIs were introduced in select video games, with them having the ability to automatically transition music created between two longer pre-recorded pieces, for example. The auto-tuning correcting errors in many pop hits also work on a similar algorithms.
But Google didn't introduce the blobs without giving them some gimmicks to make them interesting.
For example, users can use their own voice to voice the blobs. Users can run a simple pitch tracker of their voice in real-time, and then use the harmonization component from Blob Opera.
When users drag the blobs, the blob with the highest voice will always take the melody while the bassier blobs will do the harmonious groundwork for their star soloist.
What makes the blobs smart, is the way the AI harmonizes the voices in real-time to any new directions.
This kind of feat requires a large dataset of existing harmonies, fed into machine-learning algorithm that can find patterns in that data. To do this, Blob Opera uses a second machine learning model to make the blobs respond to and harmonize users input in real time.
This kind of capability could change the AI music revolution.
Such software is many years away. To really utilize the technology, many more researches about behavioral and neuroimaging modelling are needed.
For Google, the four singing blobs are just the start, and it is its way to end the sorrowful 2020 with a little joy.