Data Center Trends: Sustainability

Richard Burcher, Liquid Cooling Global Product Manager, investigates the impact of High-Density Computing on cooling technology

High-Density Computing Chip

High-Density Computing, involving advanced computer chips and next-generation GPUs, are forcing changes. All elements of the thermal management system in the data center are connected and need to be viewed as a system. Airedale by Modine can collect all the heat produced at the server from the chip and reject or repurpose that heat right the way through to the outdoors.

The message here is one of High-Power Density, increasing Thermal Loads or Thermal Design Power (TDP) at the chip, increasing rack densities, and placing strain on heat dissipation at the chip within the server. 

The world is transitioning with technology and digitalisation. We are seeing accelerated traffic growth. There is an ongoing transition from general purpose, traditional computing applications, using Central Processing Unit Servers (CPUs) for workloads that have moderate asks like communications, data storage and processing; and now, we are transitioning to more accelerated, high-density computing and Generative AI, developing AI language models, Machine Learning and Big Data Analytics. 

These high-density computing applications require higher complexity, workload, and strain. They require accelerated computation and use Graphical Processing Units or GPUs, the type of which can run to 500 watts plus, with an onward trajectory to 1,000 watts (1kW) generated at the chip, in the ranges of 80˚C – 90˚C (176˚F – 194˚F) surface temperature, at the chip, with the majority of these GPU shipments going to hyperscale and Cloud Services Providers (CSPs). 

According to Omdia, Nvidia shipped nearly half a million H100 and A100 GPUs in 3Q23 and was expected to cross the half-a-million mark in 4Q23. If you review the scale being ordered by the top players, at approximately 700 watts per chip of TDP, you can estimate that it will generate 105MW per player. That figure does not consider additional heat-generating IT components. So, with this transition as mission-critical data center applications and use cases become more central, there comes the need for more advanced, innovative cooling and power – it is now more integral.

With this increased density, you must be able to take the high-power densities away and reject the heat from the chip right the way to the external heat rejection outside the facility, and this is where liquid cooling begins to play. Airedale by Modine see from the market that systems can and will be hybrid. When looking at AI, there is often a perceived idealism that 100% will use air-cooled facilities and 100% will use liquid-cooled facilities. Airedale by Modine is seeing a hybrid approach and a mix of compute density in the data center; customers are looking at facilities with a mix of purposes, both traditional computing and more high-density computing and developing dedicated High-Performance Computing (HPC) in halls/zones, where air cooling exists, alongside liquid cooling.

In some instances, there is a mixed methodology of hybrid air and liquid-cooled systems; for example, where you have a Direct-to-Chip based Cold Plate system, in a rack, you will likely dissipate ~70–75% of that heat at the chip, meaning there is an additional ~25-30% of that heat to reject into the air from additional IT components, which needs management via CRAC/CRAH/OnRak (Rear Door Heat Exchanger) cooling systems.

It is essential to note that there is no one-size-fits-all approach, and the solutions are application and customer-dependent. Airedale by Modine is making great strides in this area, allowing an agile and agnostic approach to the solution adopted in the server, whether that be Direct-to-Chip, 1-phase Immersion, or 2-phase Immersion, allowing Airedale by Modine to provide liquid-enabled, air-cooling and hybrid-based systems. That approach enables the creation of a best-fit solution, working with customers and tracking the entire system from internal chip requirement to external heat rejection.

This post originates from Richard Burcher’s comprehensive article, ‘Trends, Challenges, and Innovations in Data Centers‘.