Researchers at Maluuna, an artificial intelligence company acquired by Microsoft, found what happened when we score a million points at classic Atari game Ms. Pac-Man.
The original game was released in 1982, and going to a seven-digit score was regarded as an inhuman feat. The highest score that a human has ever reached in the Atari 2600 version of game was 266,330, with the perfect score of 999,990 only possible using cheat codes.
The Ms. Pac-Man game is a simple game to play, but difficult to master as the user progresses. It has been a target by AI and many researchers for years. But until Microsoft puts its hands on it with Maluuna, no humans or software has ever passed 1 million points.
The moment was however anticlimatic. "It just reset to zero, it was kind of disappointing,said Rahul Mehrotra, a program manager at Maluuba.
But the company's researchers claim that the same algorithmic techniques that reached maximum possible score in Ms. Pac-Man could help machines master more complex tasks.
For Microsoft's AI to master the game, according to Mehrotra, the software needs to learn how to balance the demands of dodging four ghosts, hunting down fruits and eating pellets, all at the same time and fast.
This new founding after mastering the game, has a long-term goal for Microsoft. Maluuba is focused in creating AI research, helping Microsoft sales team and Dynamics among other things.
Tech companies have been testing their AI with Atari games. With the research, they aim to figure out how to make machines could make sense of the real world.
Ms. Pac-Man is one of those Atari Games that has proven difficult for machines to learn. While in the original Pac-Man game experts can play with their eyes closed by memorizing the game's monsters, in Ms. Pac-Man, the ghosts and fruits move in unpredictable ways. This forces players to constantly rethink their strategy.
To master Ms. Pac-Man, Maluuba created a crowd of more than 150 learning agents.
This method, named by the Maluuba team as Hybrid Reward Architecture. Each of them work on learning how each element of the game - such as fruits, ghosts, or pellets - affect the game's score. The agents then give the information they learned to a top agent, a central decider which received suggestions from all other agents to determine what Ms. Pac-Man should do next.
The top agent considers the number of agents pushing forward as a suggestion. However, its decision is also based on the intensity. One given example was when 100 agents wanted to go right to pick up a pellet, there were three agents that wanted to head in another direction to escape a ghost. The top agent would give more weight to the three agents, a pre-caution so Ms. Pac-Man won't get caught.
