OpenAI ChatGPT's Advanced Voice Mode Gets Real-Time Vision Capability: A Christmas Present

It’s the holiday season—a time for joy, connection, and a sprinkle of tech magic.

Since OpenAI introduced ChatGPT, it has created an arms race where tech companies compete with each other for supremacy. OpenAI itself keeps on improving its AI, hoping to remain relevant, and if Sora and its late launch are excluded, the company is the key player in the generative AI trend.

In the sixth day of its '12 Days of OpenAI,' OpenAI revealed a huge update to its existing Advanced Voice Mode, by giving it visual assistance and screen sharing feature.

The long-awaited feature allows users to interact with the chatbot in different modalities.

OpenAI first teased the feature back in May, when the company unveiled GPT-4o, and discussed ChatGPT learning to “watch” a game and explain what’s happening.

In a demonstration, a user received step-by-step instructions from ChatGPT’s Advanced Voice on how to make a cup of coffee.

As the user progressed, ChatGPT provided step-by-step guidance, offering helpful tips and insights throughout the process.

In other words, the feature allowing it to interpret what it sees in real time.

This kind of interactive guidance showcases how screen-sharing and visual input can enhance everyday tasks.

This Advanced Voice Mode capability is possible due to utilizing visual recognition AI, enhancing its ability to assist with what it sees through users phone camera or screen.

And because ChatGPT is natively built on natural conversational capabilities, the update to Advanced Voice Mode allows seamless, multi-turn interactions that handle interruptions and non-linear conversations effectively.

This upgrade builds on its already impressive conversational skills, enabling smooth, natural-like interactions that support interruptions, multi-turn conversations, and even non-linear trains of thought.

The new screen-sharing and visual feature is rolling out with the latest mobile app updates starting today and continuing throughout next week.

OpenAI is making them available to all Team users and most Pro and Plus subscribers.

Pro and Plus users in Europe will receive access sooner, whereas Enterprise and Edu users can expect the update early in 2025.

And lastly, because the release is just in time for the holiday season, OpenAI is also introducing a special Santa voice.

To activate it, users can click the snowflake icon.

And because it's part of a present from OpenAI, this Santa can reset users' usage limit, which means it allows users to have more time for festive conversations.