It began as Project Loon.
Back in 2011, the project experienced its first unofficial development under incubation in X Development LLC. (formerly Google X), the semi-secret research and development facility and organization founded by Google.
After it graduated in July 2018, Project Loon spun out into its own company, called Loon.
Loon creates high-altitude balloons that are sent up to the stratosphere at an altitude between 18 km and 25 km. The balloons can then work together to create an aerial wireless network to beam internet speed up to 1 Mbit/s.
With the balloons, Google's parent company Alphabet can beam internet access to rural and remote areas, as well as regions to affected by natural disaster, or to monitor weather events, to study climate changes, and more.
But keeping balloons afloat at that high altitude means that Loon needs to make them capable of dealing with extreme weather conditions, shifting winds, and more.
These natural conditions can be tough for Loon's helium-filled balloons.
Loon engineers have created an algorithm called StationSeeker that helped keep the balloons stay on course. But high up in the sky, things can be unpredictable, and this hampered the algorithms.
To develop, test and trial new algorithms, the engineers needed resources and time. And this wasn't efficient, as they use energy, which then curbed the amount of time the balloons can remain afloat.
To counter this, according to an article published by Nature, the researchers have started applying reinforced learning to the balloons.
Reinforced learning is a system that rewards computers from the actions they do. And here, the researchers trained the algorithms using AI, to understand the optimal pursuit when dealing with unknown environment. That in order to achieve better navigation results.
"Reinforcement learning is the science of getting computers to learn from trial and error," said Marc Bellemare of Google's AI division. "With reinforcement learning we are focusing on the decision part. How do we go up or down based on that data? Not only is [the AI controller] making decisions, but making decisions over time."
According to the researchers in an article:
Using AI, Loon's balloons should be better in predicting wind speed and different height and weather conditions. Through training, the balloons should be able to raise or lower accordingly.
The algorithm steers the balloons through figure-eight movements across the sky to detect the ideal current. Since many of the world's atmosphere are not yet fully monitored for wind direction and speed, the algorithm generated 'noise' to fill in data gaps and to also include historical data.
"It's super hard to have the [balloon] network over the people who need connection to the internet and not drifting far away. What the RL [reinforcement learning] is doing for us is deciding what's the situation with the balloon, how much power does it have left, what is the best action that the balloon could do right now to stay over the person with the cellphone in their hand," explained Sal Candido, Loon's Chief Technology Officer.
To make this happen, Candido said that balloons must remain within less than 50 km from the ground stations. This is required so the balloons can reliably send and receive signals.
With the updated algorithm, Loon's balloons should also remain connected longer than before, and can also return to correct their coordinates faster than before.
"To be frank, we wanted to confirm that by using RL a machine could build a navigation system equal to what we ourselves had built," said Candido.
"The learned deep neural network that specifies the flight controls is wrapped with an appropriate safety assurance layer to ensure the agent is always driving safely. Across our simulation benchmark we were able to not only replicate but dramatically improve our navigation system by utilizing RL."