During a long AI training run, a small region of an accelerator can reach its temperature limit while the cooling water is still comparatively cool. The processor may then have to lower its clock frequency. How much of that lost computing time could be recovered by bringing the water closer to the transistors?
Researchers at Microsoft and Corintis tested one answer on an NVIDIA GH200. They etched fluid passages into the back of the graphics processor and measured how much hotter the device became than the incoming coolant. The experiment also tracked the nearby high-bandwidth memory that supplies the processor with data. To understand what either result means for AI, we need to follow the heat out of the chip and the electricity into the data centre. [1]

Where does the heat meet the liquid?
In NVIDIA’s Grace Blackwell rack systems, liquid already flows through cold plates attached to the CPUs and GPUs. Heat travels out of the device, across the thermal contact and into the plate before the liquid carries it away. Etching channels in the silicon shortens that path. A sealed cover distributes coolant through the passages, and their walls provide additional surface for heat exchange. The liquid stays separated from the electronic layers. A pump moves it to a heat exchanger, and the cooling plant must ultimately release the heat to the surroundings. [1] [4] [11]
A useful measurement is the temperature difference between the active silicon and the coolant at the inlet, divided by the power dissipated by the chip. This thermal resistance, expressed in kelvins per watt, tells us how much the silicon warms up at a given load. Georgia Tech and Microsoft etched the back of a working Intel Core i7-8700K. Compared with their conventional cold plate, they measured up to 44.4% lower junction-to-inlet thermal resistance and demonstrated higher sustained core frequencies. Here the cooler changed something a program could use: how fast the CPU could keep running. [2]
What did the accelerator and its memory gain?
On the GH200 platform, the researchers reported up to a 60% reduction in junction-to-inlet temperature rise at the GPU and approximately 35% at the high-bandwidth memory, relative to conventional cold plates. These percentages describe temperature differences. For example, if a chip were 40 degrees above its inlet, a 60% reduction in that difference would leave it 16 degrees above the inlet. This arithmetic illustrates the unit of comparison. Electricity use requires a separate measurement. [1]
During sustained matrix calculations, a cooler processor can finish sooner if temperature had been limiting its clock. A different bottleneck arises when a language model generates one token at a time and the processor waits for weights to arrive from memory. Cooling the GPU may then leave response time almost unchanged. Cooling the memory could help if its own thermal limit was restricting transfers. A convincing comparison would run the same model at the same output quality, batch size and latency requirement, then measure tokens per second and total joules per response. How memory bandwidth affects AI inference. [1]
How much electricity could a data centre save?
A retrofit at Lawrence Livermore National Laboratory illustrates why the answer depends on the building. Engineers fitted the Cabernet computing cluster with water-cooled plates on its hottest components. The remaining heat still went to room air, and both routes used the existing chilled-water plant. Using operating data, the researchers estimated 4% savings in overall data-centre energy from their site-level analysis and 3% from a component-by-component analysis. The main electrical benefit came from slowing the fans in the computer-room air handlers. This was a partial cold-plate retrofit at a working computing centre. [5]
Lawrence Berkeley National Laboratory examined a different installation. It instrumented 38 liquid-cooled servers and used two unmodified servers as a reference. The researchers combined measurements with models of alternative cooling plants. At the test’s 50% server power level, the projected reduction in total site energy ranged from 14% to 20%. A separate life-cycle assessment by Alissa and colleagues estimated 15–20% lower energy demand per virtual core per year for cold-plate and immersion scenarios compared with air cooling. That boundary includes energy beyond the facility’s electricity meter. Each study compares its own systems. The additional site-wide electricity saving from replacing an existing cold plate with channels etched in the GPU remains unquantified in the GH200 study. [1] [6] [7]
Consider an illustrative centre using 100 MWh for a fixed amount of AI work. Suppose cooling accounts for 10 MWh. If a change halves that cooling electricity, including pumps and heat rejection, and everything else stays constant, the centre uses 95 MWh: a 5% reduction at the meter. A site can spend the recovered electrical capacity on additional inference requests instead. In that case, its benefit is more useful work within its grid connection, and its electricity bill may stay much the same.
Can operators adapt the equipment they have?
A rack already fitted with cold plates has part of the necessary infrastructure: supply and return manifolds, pumps and a route for rejecting heat. Reuse depends on the coolant chemistry, inlet temperature, pressure and flow that the new device requires. Smaller passages may need finer filtration. Pumping power depends on both flow and pressure drop, so a smaller flow alone cannot establish an electricity saving. An air-cooled room would also need a liquid distribution circuit, connections at the racks and a way to detect and isolate a leak. HP and NVIDIA have explored an intermediate approach, a separately fabricated silicon microchannel plate coupled to the GPU. It moves the coolant closer while allowing the cooling structure to be made separately from the processor. [2] [4] [10] [11]
The more demanding change is at the factory. Etching a finished die must leave enough silicon for mechanical strength, and the seal must withstand thermal cycling. Microsoft and Corintis have investigated mechanical stress, clogging and erosion. TSMC separately demonstrated a CoWoS-R packaging test vehicle with silicon micropillars that dissipated over 5 kW under uniform heating, using 6 litres per minute of deionised water at 40 °C. At Peking University a much smaller heater experiment reached 3,000 W/cm² at a hydraulic pumping power of 0.9 W/cm². One experiment measures total heat over a package, the other heat per unit area. Production readiness also depends on manufacturing yield, maintenance intervals and failures observed over years of service. [3] [8] [9]
When would this help AI expand?
Where a training cluster repeatedly loses clock speed to hot spots, sustained throughput is the first outcome to measure. Where the constraint is a fixed electricity supply, the relevant result is more completed training or inference work for each kilowatt-hour drawn by the site. And where memory is stacked close to logic, the physical challenge is getting heat out of an interior layer without displacing the electrical connections that made the stack attractive. Angstrom-class process names describe a manufacturing generation. The cooling passages can remain micrometre-sized: their purpose is to carry away heat from a region containing many transistors. How backside electrical connections use the same side of the chip. [1] [8] [11]
Bringing coolant inside the silicon is justified when the extra thermal margin translates into sustained computation or lower site electricity for the same AI service, and the package can deliver that gain reliably. For a system already meeting its thermal and latency targets with external plates, the decisive test is a matched run that includes the pumps and cooling plant. A cooler chip settles the thermal question. The electricity meter and the completed workload settle the efficiency question.
Sources and scope
Evidence reviewed through 18 November 2026. The CPU and GH200 studies concern working processors. The Peking and TSMC results concern thermal test devices. The facility comparisons come from separate installations and models. The CPU, GH200, reliability and life-cycle publications share some contributors or institutional affiliations.
- Majety, S., et al. (2026). Direct to Silicon Microfluidic Cooling for Datacenters. IEEE Electronic Components and Technology Conference, pp. 926–929. GPU and high-bandwidth memory tests.
- Kochupurackal Rajan, S., et al. (2022). Integrated Silicon Microfluidic Cooling of a High-Power Overclocked CPU for Efficient Thermal Management. IEEE Access, 10, 59259–59269. Operating CPU and cold-plate reference.
- Wu, Z., et al. (2025). Jet-enhanced manifold microchannels for cooling electronics up to a heat flux of 3,000 W cm⁻². Nature Electronics, 8, 810–817. Heated test chip and hydraulic power.
- NVIDIA. DGX GB Rack Scale Systems User Guide, Hardware. Installed arrangement of CPU and GPU cold plates in the compute trays.
- Lawrence Berkeley National Laboratory and collaborators (2020). The Installation of Direct Water-Cooling Systems to Reduce Cooling Energy Requirements for High-Performance Computing Centers. ACEEE Summer Study on Energy Efficiency in Buildings. Two analyses of the Cabernet retrofit at Lawrence Livermore.
- Coles, H., and Greenberg, S. (2014). Direct Liquid Cooling for Electronic Equipment. Lawrence Berkeley National Laboratory, LBNL-6641E. Instrumented servers and modelled site energy at different loads.
- Alissa, H., et al. (2025). Using life cycle assessment to drive innovation for sustainable cool clouds. Nature, 641, 331–338. Life-cycle energy demand per virtual core per year.
- Jiang, S.-D., et al. (2026). Process Development and Thermal Characterization of Micropillar Direct-to-Silicon Liquid Cooling Solution on CoWoS-R Platform. IEEE Electronic Components and Technology Conference, pp. 546–552. Packaging test vehicle under uniform heating.
- Microsoft and Corintis (2026). Reliability of In-Silicon Etched Microfluidic Cooling Systems. IEEE International Reliability Physics Symposium. Mechanical and fluid-path reliability testing.
- HP (2025). Embedded Microfluidic Cooling for Nextgen High-Power Server Architectures. ARPA-E COOLERCHIPS programme, 4 December. Silicon cold plate coupled to a GPU.
- Microsoft (2025). AI chips are getting hotter. A microfluidics breakthrough goes straight to the silicon to cool up to three times better. Prototyping, cooling passages and integration constraints.
Explore the Horizons of Electronics collection.
Liked this post? Follow this blog to get more.