Microsoft, The First Cloud Provider With Servers Using 'Two-Phase Immersion Cooling'

06/04/2021

Servers are like heavy-duty computers. They need to be on non-stop, serving and processing users' demand.

In the world where almost everything that involves information and data is web-driven, servers demand even more electricity to run. And because newer systems are more capable and more powerful, they can generate even more heat.

And to cool them, again, they need electricity to power those fans.

Microsoft is one huge cloud service provider. After submerging its cloud data center underwater for two years near Scotland through Project Natick, plunging 864 servers and 27.6 petabytes of capacity into the water, Microsoft reported success.

And with its knowledge and experience, Microsoft is ramping things up a notch.

This time, instead of submerging its servers, it flooded them.

Instead of using air to cool the servers, or use water as a medium of cool the servers' cooling system, Microsoft directly applies liquid cooling to the servers' hardware.

Liquid-cooled data center
In a closed compartment, the servers are flooded with a special liquid. (Credit: Microsoft)

On its website post, Microsoft calls this method "two-phase immersion cooling."

To do this, the company flooded its servers with a special non-conductive liquid that poses no threat to the electronics.

Inside a closed compartment that resembles a top-load washing machine, Microsoft puts the servers inside a pool of fluorocarbon-based liquid. This liquid that has direct contact with the servers' components, boils at 50°C as it carries heat away from the servers. The vapor that goes to the surface of the liquid in attempt to escape to the air, will get cooled by the tank's lid.

The vapor will condense and return to its liquid form, and will rain down back to the compartment.

This closed-loop system boasts simplicity.

Microsoft is able to lessen the cost as the system requires no energy to move the liquid around in the tank. The system doesn't even require chillers for the condensers.

“It’s essentially a bath tub,” explains Christian Belady, vice president of Microsoft’s data center advanced development group, in an interview.

“The rack will lie down inside that bath tub, and what you’ll see is boiling just like you’d see boiling in your pot. The boiling in your pot is at 100°C, and in this case it’s at 50°C.”

Most data centers are air cooled at the present time, utilizing outside air to be sucked inside a cooling system in order to drop it to temperatures of under 35°C.

Traditionally, the cooling system utilizes the vanishing method, or also known as marsh cooling. But to do that, it still requires a ton of water simultaneously.

This liquid bath tub method not only can lessen the amount of electricity needed for servers to be cooled, as it can also lessen the required amount of water.

“It potentially will eliminate the need for water consumption in data centers, so that’s a really important thing for us,” said Belady. “It’s really all about driving less and lower impact for wherever we land.”

“Our goal is to get to zero water usage,” added Belady. “That’s our metric, so that’s what we’re working towards.”

This should work towards Microsoft's goal, which pledges to replenish much more water than it utilizes for its worldwide activities by 2030.

Another advantage of using this liquid coolant, is allowing Microsoft to put the servers more tightly together. Traditionally, servers need a certain amount of space between them when they are air-cooled. But using this liquid bath and shower method, Microsoft can pack the servers closer together, saving much space.

The cooling method essentially floods the servers inside a special liquid, and allows the liquid to boil as it cools the components. (Credit: Microsoft)

While trialing this method, Microsoft is studying the reliability implications of this cooling system, and and what types of burst workloads it could even help with for cloud and AI demand.

“We expect much better reliability. Our work with the Project Natick program a few years back really demonstrated the importance of eliminating humidity and oxygen from an environment,” explained Belady.

After seeing success with Project Natick, and Microsoft had only one-eighth the disappointment pace of a land-based server farm, Belady is looking forward for this system.

“What we’re expecting with immersion is a similar trend, because the fluid displaces the oxygen and the humidity, and both of those create corrosion […] and those are the things that create failure in our systems,” said Belady.

Initially, Microsoft trials this with a small internal production workload, with plans to use it more broadly in the future.

“It’s in a small data center, and we’re looking at one rack’s worth,” said Belady. “We have a whole phased approach, and our next phase is pretty soon with multiple racks.”

According to Microsoft, the company is not the first to employ this particular cooling method: cryptocurrency miners have already been using immersion cooling for some time.

But this time, the same method is applied to Microsoft, a cloud server provider.

Previously, Microsoft has also experimented with using hydrogen fuel cells to power its data center.