What 250 kW Per Rack Actually Demands

Every hardware generation raises rack density, and the cooling decision that used to be optional is now the decision that gates the whole build. A rack that draws 250 kW rejects 250 kW of heat into a space the size of a refrigerator. Air cannot move that much energy out of that volume. This is not a matter of better fans or colder aisles. It is a limit set by the heat capacity of air and the velocity you can push through a server before it becomes a wind tunnel. Understanding where that limit sits, and what the alternatives cost, is the difference between a facility that can host next-generation GPUs and one that is obsolete the day it opens.

The Air Ceiling Is a Physics Wall

For sustained GPU workloads, air cooling has a practical ceiling around 40 kW per rack. You can push past it in short bursts or in carefully engineered rooms, but as a design basis for a facility that runs at load around the clock, 40 kW is the honest number. Between roughly 50 and 100 kW per rack, air cooling does not get harder, it stops working. The airflow required to remove the heat exceeds what the servers can tolerate, the fan power needed to move it turns into a parasitic load that wrecks efficiency, and the temperature gradients across the rack become impossible to hold within spec.

This matters because the frontier of AI compute has already walked through that wall. The NVIDIA GB200 NVL72 rack carries a thermal load in the range of 132 kW, more than three times the air ceiling. Projected next-generation density lands near 240 kW per rack, and the high-density AI regime the industry is designing for now sits at 250 kW. None of those numbers are reachable with air as the primary heat rejection path. The moment a facility commits to hosting that hardware, it has committed to liquid, whether the operator framed the decision that way or not.

Cold Plate Versus Immersion at the 250 kW Tier

Once liquid is a given, the architecture question opens. The two serious contenders at high density are direct-to-chip cold plate and immersion. They solve the same physics problem differently, and the trade-offs are not close to symmetric.

Direct-to-chip cold plate brings coolant to a metal plate mounted directly on the hottest components, the GPUs and, increasingly, other high-power silicon. The coolant absorbs heat at the source and carries it to a coolant distribution unit that rejects it to the facility loop. The servers stay recognizable, technicians service them the way they always have, and the liquid stays contained in a sealed loop that never touches the boards. Cold plate is the architecture the major GPU platforms ship against, which is why it dominates new high-density deployments.

Immersion submerges the entire server in a dielectric fluid. It handles very high heat flux and removes the fan load entirely, but it changes how everything downstream works. Serviceability is different, the fluid itself is an operating cost and a supply-chain dependency, and the mechanical and facility design has to be built around tanks rather than racks. It is a real answer for specific workloads, but at the 250 kW tier the industry has largely converged on cold plate as the default because it preserves serviceability and rides the OEM roadmap.

The Retrofit Penalty

Here is where the economics turn sharp. Adding liquid cooling to a facility that was not designed for it carries a cost premium of 40 to 60 percent over building the same liquid capability in from day one. That premium is not a rounding error, it is the difference between a competitive facility and a stranded one. The reason is structural. Day-one liquid design routes the coolant loops, sizes the coolant distribution units, provisions the facility water and heat rejection, and lays out the floor around liquid from the first drawing. Retrofit means tearing into an operating building to add all of that after the fact, working around structure, power, and tenants that are already in place.

The trap is that air-cooled facilities look cheaper on the day they are commissioned. The savings are real and they are also temporary. The moment the tenant demand moves to hardware that needs liquid, the air-cooled building faces a retrofit at a 40 to 60 percent premium, or it simply cannot host the workload and loses the tenant. A facility that skipped liquid to save capital up front pays that saving back with interest the first time density moves, which in the current hardware cycle is every generation.

The PUE Economics Over a Contract

The capex comparison is only half the story. The other half is what the cooling architecture does to power efficiency, measured as power usage effectiveness, over the life of a hosting contract. At high density, liquid cooling delivers a PUE in the range of 1.15 to 1.25. Air cooling at the same density, where it is even possible, runs a PUE of 1.4 to 1.6. That gap is the fraction of every megawatt that goes to overhead rather than to compute.

Translate that into a multi-year contract and the numbers separate. At PUE 1.2, overhead is 16.7 percent of total facility power (0.20 kW of overhead per 1 kW of IT load). At PUE 1.5, overhead is 33.3 percent of total facility power (0.50 kW per 1 kW of IT load). On the same IT load, over the same term, the air-cooled facility burns dramatically more power to deliver the same useful compute, and in a market where power is the scarce input, that overhead is a direct hit to margin and to the price the operator can offer. Liquid does not just enable the density, it lowers the cost of every kilowatt-hour of useful work for the entire life of the contract.

The Category Question

Put the pieces together and a pattern emerges. Density doubles roughly every hardware generation. The air ceiling has not moved, and it will not, because it is set by physics. The retrofit penalty for adding liquid late runs 40 to 60 percent, and the PUE advantage of liquid compounds over every contract. Each of those facts points the same direction.

So the real question is not which facilities should adopt liquid, it is whether day-one liquid design has stopped being a premium feature and started being table stakes. The retrofit math says it already has. A facility designed for air is a facility designed for the hardware that is already shipping, not the hardware that will fill it. In 2026, building for liquid from the first drawing is not the aggressive choice. It is the baseline, and everything else is a bet that density will stop climbing, which nothing in the roadmap supports.

Further reading

• Power Density at 50 kW Per Rack - what the first step past the air ceiling costs and breaks.
• The GPU Rack Density Timeline - how the industry moved from 20 kW to 200 kW per rack, and where it goes next.