DeepMind started its life as a startup what makes AI that can play games. Gone are those days.
With the advancements in technologies, DeepMind, co-founded by Mustafa Suleyman and a property of Google, has gone beyond just making AI that can play games. The company has evolved to also create artificial intelligence in as many forms as it can,
And this time, the team managed to create an AI that can write code to solve arbitrary problems its researchers gave it.
As a matter of fact, the researchers have proven that the AI is capable, by making it participate in a coding challenge.
As explained by DeepMind's researchers in a blog post:
Competitive programming can be considered an event that involves hundreds of thousands of programmers that participate in coding competitions to gain experience and showcase their skills in fun and collaborative ways.
During such competitions, participants receive a series of long problem descriptions and a few hours to write programs to solve them.
Participants are then ranked mainly based on how many problems they can solve with their programs.
In this case, the researchers at DeepMind pitted its AI in a competitive programming, hosted on Codeforces, a popular platform which hosts regular competitions.
According to the researchers, AlphaCode achieved an estimated rank within the top 54% of participants in programming competitions by solving new problems that require a combination of critical thinking, logic, algorithms, coding, and natural language understanding.
AlphaCode ended up only around the middle, meaning that it's only average.
Although far from winning the competition, this result represents a substantial leap in AI problem-solving capabilities.
Coding requires a combination of language mastery, logic, and problem-solving skills.
And program-solving abilities required to excel at these competitive programming competitions "are beyond the capabilities of existing AI systems."
The researchers at DeepMind managed to create this AlphaCode by combining advances in large-scale transformer models with large-scale sampling and filtering.
The researchers then pre-trained the model on selected public GitHub code and fine-tuned it on its relatively small competitive programming dataset.
"Then we filter, cluster, and rerank those solutions to a small set of 10 candidate programs that we submit for external assessment. This automated system replaces competitors’ trial-and-error process of debugging, compiling, passing tests, and eventually submitting," the researchers added.
In the end, the researchers managed to create AlphaCode, an AI that leverages deep learning models that excel at tasks that require “critical thinking, logic, algorithms, coding, and natural language understanding.”
In a more detail explanation, the researchers have released a pre-print about this AlphaCode.
In the modern days of technology, practically everything that has a logic board can be programmed to do at least one specific task. And to do that, there needs to be some codes to run the program.
For all this time, humans have been brains behind the codes.
While the tools programmers use can have some sorts of automation, having an Ai capable of coding is certainly a natural fit for a computer’s capabilities.
“I can safely say the results of AlphaCode exceeded my expectations,” said Mike Mirzayanov, founder of Codeforces. “I was skeptical because even in simple competitive problems it is often required not only to implement the algorithm, but also (and this is the most difficult part) to invent it. AlphaCode managed to perform at the level of a promising new competitor.”
DeepMind has created a demo web page to allow people to test this AI out.
This is not the first attempt to create an AI capable of programming.
Previously, OpenAI has is own Codex natural-language coding project, and it is powering GitHub Copilot, among others.
Thanks to large-scale language processing models, researchers have demonstrated how AIs can be made and trained to have impressive abilities.
And coding is just one of those abilities.
With researchers realizing that AIs can be taught to program, it's worth noting that AIs of this kind can still perform poorly when evaluating on more complex, unseen problems that require problem-solving skills beyond simply translating instructions into code.
This is one of the reasons why AlphaCode and others of its kind are not going to take any software engineers' jobs just yet.
Researchers suggest that the AIs may come in handy when it comes to automating basic tasks.
Further reading: Google's DeepMind Develops An AI That Uses Imagination In Its Decision-Making Process