Meta Unveils 'Meta Gen' AI For Creating Realistic, Customizable Videos With Sounds

Meta Movie Gen

Reality is like a handwoven tapestry, made of threads, with each stitch representing a moment lived, felt, or touched.

AI-generated content, on the other hand, is like a digital painting. It can be beautifully precise, creatively rich, but created from pixels. One is like born from nature and whatever incidents happen within a timeframe, while the other is crafted from code that compiles to become pixels.

While AI-generated content mimics the patterns of reality, it lacks the imperfections and depth of the woven threads.

But as AI becomes smarter and more powerful, this line that differentiate them blurs.

The two worlds somehow become less and less distinguishable.

After OpenAI introduced ChatGPT, and following the trend that created the hype, which created an arms race between tech companies, what looks real may not be real, because fakery can now be more realistically real.

This time, Meta, the parent company of Facebook, Instagram, WhatsApp and some others, revealed that it has developed an AI model it calls 'Movie Gen':

In a blog post, Meta said that "Meta Movie Gen could usher in a new AI-enabled era for content creators."

"Whether a person is an aspiring filmmaker hoping to make it in Hollywood or a creator who enjoys making videos for their audience, we believe everyone should have access to tools that help enhance their creativity. Today, we’re excited to premiere Meta Movie Gen, our breakthrough generative AI research for media, which includes modalities like image, video, and audio."

"This work is part of our long and proven track record of sharing fundamental AI research with the community."

According to Meta, this tool is capable of creating realistic video from just text prompt.

This isn't exactly new, because a bunch of AI tools can do this as well.

But what makes Movie Gen so different that it's on a different league, is the fact that it can also generate audio clips that correspond to the video clips it create.

This allows it to position itself as more than just a powerful competitor to OpenAI and some others.

According to Meta, Movie Gen has already produced sample clips showing animals swimming and surfing, as well as videos where people’s real photos are transformed into scenes depicting actions such as painting on a canvas.

The model can also generate background music and sound effects that sync seamlessly with the videos, and it allows users to edit existing footage.

One demonstration showed Movie Gen adding pom-poms to the hands of a man running alone in the desert, while another video modified a dry parking lot into one filled with splashing puddles as a man skateboarded through.

Videos created by the model can be as long as 16 seconds, with audio clips extending up to 45 seconds.

As the entertainment industry grapples with the growing potential of AI-generated video technology, Meta’s announcement comes at a time when Hollywood has been exploring ways to incorporate such tools into filmmaking.

In February, the Microsoft-backed OpenAI introduced its own generative AI product, Sora, which demonstrated the ability to create feature-film-like videos based on text prompts. While some in the entertainment field see these tools as opportunities to speed up and enhance filmmaking.

One of the first was Toys "R" Us, which created an advertising generated entirely with the AI.

Since then, OpenAI has been engaging with Hollywood executives and agents throughout the year, exploring potential collaborations involving its Sora tool.

But concerns also arise, with an increasing number of parties are alarmed about the fact that the AI models could be trained on copyrighted works without permission.

Not to mention about malicious actors who could use these tools for creating increasingly realistic deepfakes, citing risks in countries like the U.S., Pakistan, India, and Indonesia.

In its case, Meta has stated that its Movie Gen AI is unlikely to be made publicly available to developers, unlike its Llama series of large language models.

The company explained that it evaluates risks on a case-by-case basis for each model and refrained from commenting on the specific risk assessment for Movie Gen. Instead, Meta plans to collaborate with the entertainment industry and content creators and intends to integrate the tool into its own platforms by next year.

In its accompanying research paper, Meta shared that Movie Gen was developed using a combination of licensed and publicly available datasets.

Published: 
08/10/2024