AIs have been used to diagnose illness, detect fraud and play complex games. This time, an AI is capable of passing middle and high school exam tests with brilliant marks.
Created by the Allen Institute for Artificial Intelligence (AI2), the AI is called 'Aristo'.
Despite its struggle for tackling problems that require reasoning and commonsense, it was capable of finding subtle and implicit meanings in written and spoken language.
In this instance, Aristo is capable of scoring above 90% on an 8th grade science test, and 80% on a 12th grade exam.
Aristo shows how AIs are becoming remarkable in language and logic skills.
In comparison, in 2015, some 700 computer scientists competed for a $80,000 price to develop an AI that could merely pass this kind of test. Aristo here, is considerable an achievement, given that previous AI models were only capable of scoring no more than 60% on the same tests.
Aristo was built using neural-network technology called 'BERT', originally developed by Google.
BERT is a method of pre-training language representations, meaning that researchers can train a general-purpose "language understanding" model on a large text corpus, and then use that model for downstream Natural Language Processing tasks.
BERT can be instructed to read thousands of articles and books, through which it will learn the patterns and mechanics of language, to then capable of understanding sentences, and can fill missing words by correctly determine what it was.
Aristo by AI2 on the other hand, was taught by reading numerous questions and answers that might be found on multiple-choice exams. Over time, the AI was able to learn logical patterns in the test.
One of the multiple question the AI took was logic-based questions like this:
Which change would most likely cause a decrease in the number of squirrels living in an area?
(1) a decrease in the number of predators
(2) a decrease in competition between the squirrels
(3) an increase in available food
(4) an increase in the number of forest fires
Aristo answered (4) in this question, the correct answer.
While it excels at several criteria, like capable of interpreting language to understand multiple choice questions, it is not really capable of featuring answers in illustration or graph.
According to the research paper for Aristo, the authors noted that, science tests “explore several capabilities strongly associated with intelligence, including language understanding, reasoning, and use of common-sense knowledge.”
Aristo's ability to correctly answer questions in a way that excel humans, represents a significant jump from previous AI systems, like AlphaGo.
In 2015, the Google-developed AlphaGo and made it the first computer to defeat a professional human Go player in a match without handicaps. Impressive indeed, but winning Go is a matter of learning and exploiting a fixed set of rules.
In contrast, successfully learning and applying logic to answer questions about the real world, as Aristo does, is another pursuit altogether.
This can make Aristo a benchmark to see whether an AI model can extract meaning beyond what’s explicitly mentioned in text, but doesn't mean that computers are already as smart as some humans (teenagers).
"This has significant business consequences," said Oren Etzioni, a former University of Washington professor who oversees the Allen Institute, to The New York Times. "What I can say — with complete confidence — is you are going to see a whole new generation of products, some from start-ups, some from the big companies."
"Although Aristo only answers multiple choice questions without diagrams, and operates only in the domain of science, it nevertheless represents an important milestone towards systems that can read and understand."
In other words, Aristo only represents a progress in the AI industry, where people are racing and developing machine-learning agents that becoming increasingly better.
It also suggests that soon, humans may see some further striking improvements in AI-based technology.