Unlike reading a bunch of text, viewing a video can cross the language barrier. But for Google, that isn't enough.
Google has Google Translate, which can automatically translate lots of languages. But translating videos is a very different procedure, and it requires more than just a "translator," and it needs more processes.
Google owns YouTube, and to make it a better place for its multitude of users who speak different languages, the company is introducing a tool, which came from its Google’s in-house incubator Area 120.
At VidCon, the company revealed that YouTube has partnered with Google's AI-powered dubbing service 'Aloud.'
What it does, is helping creators to dub their videos into other languages using AI, in a straightforward process that is easy to follow.
Earlier this 2023, YouTube introduced support for multi-language audio tracks, which allows creators to add dubbing to their new and existing videos.
This allows them to reach a much wider international audience.
According to YouTube, as of June 2023, creators have dubbed more than 10,000 videos in over 70 languages.
Previously, creators had to partner directly with third-party dubbing providers to create their audio tracks. This however, can be time-consuming, and also expensive.
Aloud on the other, allows creators to dub videos at no additional cost.
This is because Aloud can eliminate the time and the expense needed to dub a video the usual way (with human translators and narrators).
The tool works by first transcribing a video, to then give the creator a transcription that they can review and edit.
After that, it can then translate and produce the dub.
"Dubbing used to take weeks worth of effort and a large budget. But with Aloud, you only need a few minutes," said Google in a blog post.
Aloud was first introduced back in 2022, as an AI-powered dubbing tool. This time out of the incubator, YouTube is testing it with hundreds of creators, explained YouTube’s VP of Creator Products, Amjad Hanif.
Initially, the tool is available to only Bahasa Indonesia, Hindi and some more.
Google plans to expand this in the future, and release it to more users.
Hanif added that YouTube is “working to make translated audio tracks sound like the creator’s voice, with more expression and lip sync.”
And because the tool uses AI, and that Google is also working on generative AI, YouTube confirmed that the AI would allow Aloud to launch features like voice preservation, better emotion transfer and lip reanimation.