Microsoft is now submerging servers in liquid baths

Microsoft is starting to submerge its servers in liquid to improve its performance and energy efficiency. A rack of servers is now being used for production loads in what appears to be a liquid bath. This immersion process has been around in the industry for a few years, but Microsoft claims to be “the first cloud provider that performs two-phase immersion cooling in a production environment”.

Cooling works by completely submerging the server racks in a specially designed non-conductive fluid. Fluorocarbon-based liquid works by removing heat as it directly hits the components and the fluid reaches a lower boiling point (122 degrees Fahrenheit or 50 degrees Celsius) to condense and fall back into the bath like a liquid in the rain. This creates a closed-loop cooling system, reducing costs, as no energy is needed to move the liquid around the tank and no cooler is needed for the condenser.

Image: Microsoft

“It’s essentially a bathtub,” explains Christian Belady, vice president of Microsoft’s advanced data center development group, in an interview with The Verge. “The shelf will be inside the bathtub, and what you will see is boiling just as you would see it boiling in your pot. The boil in your pan is at 100 degrees Celsius, and in this case, it’s 50 degrees Celsius. “

This type of liquid cooling has been used by cryptocurrencies in recent years to extract bitcoins and other cryptocurrencies. This method has inspired Microsoft to test its use in recent years, using it to test spikes in cloud demand and intensive workloads for applications such as machine learning.

Most data centers are air-cooled now, using outside air and cooling it down to temperatures below 35 degrees Celsius using evaporation. This is known as swamp cooling, but it uses a lot of water in the process. This new liquid bathing technique was designed to reduce the use of water. “This will potentially eliminate the need for water consumption in data centers, so this is really important for us,” says Belady. “In fact, it’s about driving less and less impact wherever we land.

Image: Microsoft

This set of servers also allows Microsoft to package the hardware more compactly, which should reduce the amount of space needed in the long run compared to traditional air cooling. Microsoft is testing this initially with a small in-house production workload, with plans to use it more widely in the future. “It’s in a small data center and we’re evaluating the value of a rack,” says Belady. “We have a fully phased approach and our next phase will be soon with several racks.”

Microsoft will primarily study the reliability implications of this new cooling and what types of peak workloads it could even help with for cloud and AI demand. “We expect much better reliability. Our work with the Project Natick program a few years ago really demonstrated the importance of removing moisture and oxygen from an environment, ”explains Belady.

Image: Microsoft

The Natick Project saw Microsoft sink an entire data center to the bottom of the Scottish sea, plunging 864 servers and 27.6 petabytes of storage into the water. The experiment was a success, and Microsoft had only an eighth of the failure rate of a terrestrial data center. “What we expect with immersion is a similar trend, because the fluid displaces oxygen and moisture, and both create corrosion … and these are the things that create failure in our systems,” says Belady.

Part of that work is also related to Microsoft’s environmental promise to fight water scarcity. The company has pledged to replenish even more water than it uses in its global operations by 2030. This includes Microsoft using a local rainwater collection system in its offices and collecting condensation from air conditioners for water stations. However, Microsoft withdrew nearly 8 million cubic meters of water from municipal systems and other local sources in 2019, compared to just over 7 million in 2018.

Microsoft’s effort to address water use will be extremely challenging, given its tendency to use more water, but projects like two-phase immersion will certainly help if implemented more broadly. “Our goal is to achieve zero water use,” says Belady. “This is our metric, so that’s what we’re working on.”