In the world where AI is given more and more importance in everyday life, and as more the technology is being used, the more it wows the world.
While AI allows computers to work beyond their original programming, one of the greatest issues about this technology, is how to make a fully autonomous agent to work continuously in open-ended environments.
Due to the practically limitless possibilities, researchers have yet to understand how to create AI that learns as it improves, and improves as it learns.
Traditional methods rely on fundamental actions to train models through reinforcement learning and imitation learning, can make methodical investigation, interpretability, and generalizability difficult.
But thanks to the rapid advances in large language model (LLM) based agents, researchers can use the technology to learn the world information, by encoding it, in order for the LLM to develop consistent action plans or executable policies.
And this time, developer MineDojo is building an open-ended embodied AI agent with an internet-scale knowledge, by letting it play Microsoft's Minecraft.
On its website, the developer said that:
" [...] Voyager interacts with GPT-4 via blackbox queries, which bypasses the need for model parameter fine-tuning. The skills developed by Voyager are temporally extended, interpretable, and compositional, which compounds the agent's abilities rapidly and alleviates catastrophic forgetting. Empirically, Voyager shows strong in-context lifelong learning capability and exhibits exceptional proficiency in playing Minecraft."
The AI is consists of three key components:
- An automatic curriculum that maximizes exploration.
- An ever-growing skill library of executable code for storing and retrieving complex behaviors.
- A new iterative prompting mechanism that incorporates environment feedback, execution errors, and self-verification for program improvement.
The team makes use of GPT-4, which is the brain behind OpenAI's ChatGPT.
Voyager is essentially a framework built on the popular Minecraft game, where the agent is tasked with thousands of open-ended and language-prompts, and sent to freely explore a procedurally generated 3D world with diverse terrains to roam, materials to mine, tools to craft, structures to build, and wonders to discover.
Voyager, is claimed to be the first LLM-powered embodied lifelong learning agent in Minecraft.
Through its continuous learning process, it is always exploring new worlds, acquiring new skills, and making discoveries without any help from humans.
Voyager’s talents grow fast thanks to the compositional synthesis of complex skills, and that Its exploration progress and the agent’s current state are factored into the automatic curriculum.
The approach makes Voyager capable of preventing the catastrophic forgetting that plagues other forms of continuous learning
As a result, Voyager is able to solve increasingly more difficult tasks, even without human intervention.
The end result, is an AI that is able to obtain 3.3x more unique items, travels 2.3x longer distances, and unlocks key tech tree milestones up to 15.3x faster than prior state-of-the-art AI.
According to the developer, Voyager is able to do this by utilizing all of its learned skill library to solve novel tasks from scratch, while other techniques struggle to generalize.
With “discovering as many different things as possible” as its overriding purpose, the GPT-4-powered AI creates the course outline by itself, and build its knowledge from there.
Another way of explaining this, the idea is a novel strategy, in which the AI operates inside a certain context.
Voyager’s skill library is built over time from the active programs that contribute to a successful task resolution. The embedded description of each program serves as an index that can be retrieved in future analogous instances.
The goal of the research, is to find out how to create a LLM that can develop the right action code on the spot, and revise its code whenever it gets it wrong.
Read: OpenAI Announces 'GPT 4', A More Powerful, More Creative, But Safer GPT
While Voyager shows the potential of LLMs, there are issues.
First of, is the restriction and the price of labor. Since the AI requires GPT-4, it costs more than GPT-3.5.
Then there is the fact about inaccuracies, like for example, when it failed to interpret a spider string as evidence of a successful spider-killing attempt.
After that, the AI also suffers from hallucinations. And in its case, it can also suffers from code creation hallucinations, where it attempts to use a function not supported by the APIs for the specified control primitives.
Regardless, the researchers are optimistic that future updates to the GPT API models and cutting-edge methods for fine-tuning open-source LLMs should eliminate these drawbacks.
Voyager’s capacity for lifelong learning is impressive, thanks to its ever-expanding library of reusable, interpretable, and generalizable action programs for performing individual tasks.
This is why the AI might be used as a starting point to create effective generalist agents without fine-tuning the model parameters.
The researchers at MineDojo has released the AI as an open-source project on GitHub.
