Technology has come a long way, and AI is one of the advancements that create possibilities like never before.
One example, is animations in films and videos games, where actors and actresses have to use motion capture headset with specialized cameras and sensors to capture their facial movements. This gear included a head-mounted camera that recorded facial expressions in high detail. The equipment is covered in markers, often small dots placed on the face.
These markers allow the computer to map the actors and the actress expressions onto the digital models of their characters, frame by frame.
The process is called facial motion capture or facial performance capture.
AI is changing this, and it's able to achieve a similar feat, but without the contraption.
Introducing, Act-One. A new way to generate expressive character performances inside Gen-3 Alpha using a single driving video and character image. No motion capture or rigging required.
Learn more about Act-One below.
(1/7) pic.twitter.com/p1Q8lR8K7G— Runway (@runwayml) October 22, 2024
Runway AI, the American company headquartered in New York City that specializes in generative artificial intelligence research and technologies, focuses on creating products and models for generating videos, images, and various multimedia content.
And here, it has introduced what it calls the 'Act-One' feature, which is a way to "generate expressive character performances using simple video inputs."
In a web page, the company wrote:
"Act-One can create compelling animations using video and voice performances as inputs. It represents a significant step forward in using generative models for expressive live action and animated content."
Read: Runway Introduces 'Gen-3 Alpha' Image-To-Video AI: Focusing On Speed And Quality
Act-One allows you to faithfully capture the essence of an actor's performance and transpose it to your generation. Where traditional pipelines for facial animation involve complex, multi-step workflows, Act-One works with a single driving video that can be shot on something as… pic.twitter.com/Wq8Y8Cc1CA
— Runway (@runwayml) October 22, 2024
The goal is to be able to capture "the essence of a performance."
"The goal is to transpose an actor's performance into a 3D model suitable for an animation pipeline. The key challenge with traditional approaches lies in preserving emotion and nuance from the reference footage into the digital character."
With Act-One, actor and actress can have their expression recorded and portrayed by their characters, without requiring extra equipment.
Without the need for motion-capture or character rigging, Act-One is able to translate the performance from a single input video across countless different character designs and in many different styles.
(3/7) pic.twitter.com/n5YBzHHbqc— Runway (@runwayml) October 22, 2024
"Act-One is able to translate the performance from a single input video across countless different character designs and in many different styles," wrote Runway on its X account.
Initially, Act-One is focused "mostly" on the face "for now," according to Cristóbal Valenzuela, co-founder and CEO of Runway.
Runway’s approach offers significant advantages for animators, game developers, and filmmakers alike.
The model is able to accurately captures the depth of an actor or actress performance, while remaining versatile across different character designs and proportions.
This opens up exciting possibilities for creating unique characters that express genuine emotion and personality.
One of the models strengths is producing cinematic and realistic outputs across a robust number of camera angles and focal lengths. Allowing you generate emotional performances with previously impossible character depth opening new avenues for creative expression.
(4/7) pic.twitter.com/JG1Fvj8OUm— Runway (@runwayml) October 22, 2024
Act-One’s standout feature is its ability to produce cinematic-quality, realistic outputs from multiple camera angles and focal lengths, giving creators unprecedented flexibility.
This versatility allows for emotionally resonant character performances, capturing details that previously required costly equipment and complex workflows.
With Act-One, the emotional depth and unique style of an actor/actress performance can now be faithfully translated, even in challenging scenes.
This shift brings characters to life in fresh, impactful ways, opening up opportunities for richer storytelling across both live-action and animated formats.
Previously, Runway's tools, such as Gen-2 and Gen-3 Alpha, enabled video-to-video AI conversion, allowing users to upload footage and apply AI-driven “reskinning” effects.
Act-One, however, is purpose-built for advanced facial mapping and effects, setting a new standard in character-driven visual storytelling.
A single video of an actor is used to animate a generated character.
(5/7) pic.twitter.com/F3X5VpGkEN— Runway (@runwayml) October 22, 2024
Since the generative AI boom that started soon after OpenAI introduced ChatGPT, the race that is becoming more intense is seeing a lot of tools that do pretty much the same thing.
But Runway's focus differs than most.
Runway, which quickly becomes one of the leading AI video platforms, shows how AI can help enhance filmmaking in general through character consistency, which is something most AIs lack.
In other words, Act-One is like a new approach to AI video generators, which allows a more modern puppeteering: it allows users to film themselves or have an actor/actress perform and then use AI to completely change the way they look.
It's able to accomplish this by allowing users to upload a video along with a control image, and let the AI work afterward.
As AI video tools get better in reading human motion, lip-synching and character development, Runway with Act-One manage to tackle one of the biggest AI weaknesses.
"The consistency and performance is unmatched with Act-One," explained Valenzuela.
With Act-One, eye-lines, micro expressions, pacing and delivery are all faithfully represented in the final generated output.
(6/7) pic.twitter.com/R7RX6UhPEQ— Runway (@runwayml) October 22, 2024
In a post on X, Valenzuela noted a shift in how the industry approaches generative models.
"There has been an obsession with technology, with people trying to figure out what to do with models. Now is the time to start from the other side and work backwards to the technology. The why matters. Finally."
Generate expressive performances using a single driving video and character image with Act-One.
Learn how in today's Runway Academy. pic.twitter.com/BngQA2xuk1— Runway (@runwayml) October 28, 2024
According to Runway, Act-One charges 10 credits per second with a minimum of 5 seconds.
What this means, a video that is 5 seconds long will require 50 credits.
After the 5 second minimum, each additional second is charged 10 credits, with partial seconds accounted for and rounded up to the nearest decimal. In an example, a 5.6 seconds video costs 56 credits.
Users can hover over the duration modal to see the calculated credit cost before generating.
They can click on the Generate button after confirming that they're content with the selected inputs and credit costs.
"Your video will begin processing in your current session, where each video will be available for review once complete," explained Runway in a dedicated help page.
Access to Act-One will begin gradually rolling out to users today and will soon be available to everyone.
To learn more, visit https://t.co/qJjFdr5W54
(7/7)— Runway (@runwayml) October 22, 2024