AWS developed a custom liquid cooling system to cool Nvidia’s next-gen AI GPUs

Amazon’s cloud computing division, Amazon Web Services (AWS), has devised a bespoke cooling system to keep the temperatures of Nvidia’s powerful new artificial intelligence (AI) chips in check. 

On Wednesday, AWS announced that it developed the In-Row Heat Exchanger (IRHX), its custom liquid-cooling system designed for data centers with high-performance Nvidia GPUs.

Such chips, which are used to train large AI models like the chatbots or image generators so popular today, are among the power-hungriest pieces of hardware in the world. Their rising popularity in AI workloads has strained traditional cooling solutions, particularly air-based cooling.

Rather than leaving it up to the market to deliver a scalable cooling solution, Amazon had to get creative and solve it. The IRHX is suitable for retrofit and new data center applications without requiring a wholesale redesign. It does this by circulating chilled liquid close to the servers’ rows to remove heat from the tightly packed GPUs.

Dave Brown, Vice President of Compute and Machine Learning Services at AWS, explained that standard cooling solutions were not viable for their needs. He said these options would have wasted too much data center floor space and used water inefficiently. While such systems might work for a few cabinets at smaller providers, he noted they lacked the liquid-cooling capacity required to support AWS’s scale.

AWS launches P6e instances featuring Nvidia Blackwell GPUs

AWS has also just introduced P6e instances that leverage Nvidia’s GB200 NVL72, a dense, supercomputing platform containing 72 Blackwell GPUs in one rack. These are designed to cope with the computationally intensive nature of huge AI models and generative AI tasks.

Until now, only companies like Microsoft and CoreWeave have offered this next-level GPU cluster. And now AWS customers can access the newest and most advanced custom GPU machine learning training infrastructure available in the cloud, powered by the latest-generation, water-cooled NVIDIA A100 Tensor Core GPUs.

The IRHX keeps these pockets of clusters at safe temperatures, providing optimal performance without overheating. By baking the IRHX directly into its data center design, AWS can avoid waiting to retrofit entire structures for liquid cooling or paying for costly construction.

In his announcement of the launch of the P6e, Brown noted that by combining the GB200 NVL72 system with Amazon’s IRHX, customers can leverage unmatched computing power at scale. It will also allow developers, researchers, and companies to train much larger AI models more quickly and efficiently than they could in the past.

Amazon strengthens its lead in cloud infrastructure

The push to in-house its cooling tech at the in-progress data center reveals even more about Amazon’s broader play to own more of its infrastructure. In recent years, AWS has spent heavily developing its chips, storage systems, and networking gear to power its cloud services.

These advancements enable Amazon to mitigate reliance on third-party suppliers further and strike a balance between operational performance and cost.

This approach has paid off. In the first quarter of 2025, AWS notched its highest operating margin since the unit was created and is now the chief engine of Amazon’s overall profitability. The IRHX launch expands AWS’s innovation leadership and infrastructure footprint in the cloud industry.

Other tech titans are also doing likewise. Microsoft, for example, built its own AI chips and custom cooling system, Sidekicks, to go with them. Google and Meta are also investigating ways to construct hardware and systems tailored to AI workloads.

However, Amazon has a crucial advantage — its sprawling global footprint of data centers and years of experience building and deploying custom hardware at scale. The IRHX could add to that by streamlining its AI-ready infrastructure, making it more efficient, sustainable, and scalable.

Cryptopolitan Academy: Tired of market swings? Learn how DeFi can help you build steady passive income. Register Now