This AI Concept From Meta Lets Users Build Metaverses By Describing Them

The internet has came a long way. And it's still growing, as it is also evolving.

Meta is a company rebranded from Facebook. What started as a project launched from a dorm room, has conquered the social media sphere and dominates the tech industry through its various products.

With CEO and founder Mark Zuckerberg envisions that the future of the internet is inside the metaverse, he showed off an AI concept that is capable of creating new virtual worlds using just voice commands.

He called this AI the 'Builder Bot.'

Traditionally, developers would have to interact with various interfaces and type certain codes to program. But using Builder Bot, developers can create new metaverses by simply describing out loud what they want.

In the demo, Zuckerberg showed off how he could ask for a beach, and then create for a specific type of cloud, to then add props like trees and a picnic.

His colleague asked for specific ambient background sounds, while Zuckerberg asked the stereo Builder Bot created to play a music genre.

In short, Zuckerberg showed how his avatar could enter a blank metaverse and start creating a whole new scenario from nothing.

"You’ll be able to create nuanced worlds to explore and share experiences with others with just your voice," Zuckerberg said.

Meta said that Builder Bot was made by self-supervised learning (SSL), a method to train AI models using unlabeled sample data.

This method of training can be regarded as an intermediate form between supervised and unsupervised learning.

The neural network learned in two steps.

First, Meta tasked it to solve problems based on pseudo-labels which help to initialize the network weights, and second, having the AI to learn with supervised or unsupervised learning.

According to Meta in a blog post, "self-supervised learning enables AI systems to learn from orders of magnitude more data, which is important to recognize and understand patterns of more subtle, less common representations of the world."

Meta’s researchers said that relying on labelling data is a bottleneck to developing more generalized AI. With self-supervised learning, AIs should be able to get a deeper understanding of a concept from far less data that doesn’t need to be labeled.

Facebook has been developing ways to train AIs using self-supervised learning method, and has particularly found practical application in audio processing for speech recognition.

This time, Meta is picking up Facebook advances, to create this Builder Bot.

And with Horizon Worlds, Meta is already trying to lower the barriers to creating VR experiences.

Read: Meta Introduces 'Horizon Worlds' As Its First Foray Into The Metaverse Business

Digital assistants are the AIs born from the advances of voice-based human-computer interaction.

The development of the technology allows computers to understand people, and through the flow of conversation, allows devices to make people's lives easier. Digital devices have shown great advances on phones, smart homes and other devices.

The thing is, the AIs that power digital assistants still leave a lot to be desired in terms of understanding requests, speed and accuracy of information.

"Assistants today — whether via voice or chat— are generally underwhelming," said Meta conversational AI tech lead Alborz Geramifard. "There are several reasons why, starting with how they are engineered. We’re sharing the challenges developers and engineers face when attempting to build useful assistants and how we can navigate these challenges as we build for the metaverse."

This time, Meta generally wants to develop a much better digital assistant than Apple's Siri, Amazon's Alexa or Google Assistant.

While Meta hasn’t picked out a final name for its AI, Zuckerberg said that Meta wants its voice assistant to be more intuitive: picking up contextual clues in conversations, along with other data points that it can collect about its users.

It is described that this AI from Meta for the metaverse should be able to understand facial expression and hand gestures.

"To support true world creation and exploration, we need to advance beyond the current state of the art for smart assistants," Zuckerberg said. "When we have glasses on our faces, that will be the first time an AI system will be able to really see the world from our perspective — see what we see, hear what we hear, and more. So the ability and expectation we have for AI systems will be much higher."

When Zuckerberg introduced this AI, he described it as a "concept," warning that "there are a lot of challenges we still need to solve."

For example, there are delays that can happen between giving a command and seeing it actualized. What's more, Meta didn't detail how much of the demo is real, nor saying whether the 3D models exist in the library or were dynamically generated on the fly.

Beyond that, there are lots more to ask.

For example, there are questions regarding the AI generation process that can pose unique moderation problems if users ask for offensive content, or when the AI reproduces human biases and stereotypes about the world.

After that, there are issues concerning privacy.

Obviously, people need to share data to Meta in order for this AI to work. Since this AI is meant to be able to pick up sensitive information, like eye movements, facial expressions, and body language, there are questions about ethics.

With this kind of input, what kind of predictions AI can make? How will Meta benefit from knowing that kind of data?

The more information users share, there needs to be transparency in the way this AI works. Users need to have more control over how their data is used.

Aside from the privacy concerns, there are questions regarding the accessibility of this AI and metaverse, and how Meta can deal with its users' well beings.

Already, Meta faces blames from Frances Haugen, a whistleblower who said that Meta prioritizes profit above anything else. Then, inside Horizon Worlds, some women have complained about experiencing sexual harassment in the metaverse, and that a researcher found that the metaverse is not for children.

Addressing the concerns, Meta said that is committed to solve them.

Zuckerberg also said that the company is working with human rights, civil rights, and privacy experts to build “systems grounded in fairness, respect, and human dignity.”

But given the company’s track record of privacy breaches, people should be skeptical.