Energy efficiency and miniaturization thanks to water-cooled chips

March 1, 2011

Energy consumption poses a critical challenge in the development of next-generation supercomputers and IT systems. Within the next 10 years, IBM scientists and developers aim to build computers featuring exascale computing performance, but with an absolute energy consumption that is not much higher than that of today’s largest systems. Exascale computers are capable of reaching a performance of one ExaFLOP/s, which corresponds to 1018 floating point operations per second. This is about 300 times faster than today’s fastest supercomputer.

New water-cooling technologies that wick off heat right where it is being generated—directly at the chip—offer a promising route to boost significantly the overall energy efficiency of computers. At this year's CeBIT, IBM is presenting its first so-called hot-water-cooled systems, which will provide a sneak preview of future innovations: Supercomputers the size of sugar cubes.

First successful projects

Cooling with hot water combines several advantages: It eliminates the need for energy-intensive coolers, which greatly reduces overall . In addition, the removed heat can be reused directly for such purposes as to heat buildings or as process heat. This will improve the CO2 balance of computers and data centers significantly compared to similar air-cooled systems.

In a pilot project, IBM scientists and engineers from Zurich, Switzerland, and Böblingen, Germany, built a revolutionary hot-water-cooled supercomputer for ETH Zurich, a world-renowned technical university. Dubbed Aquasar, the novel is cooled with hot water instead of cold air, and the dissipated heat is used to heat the ETH campus. The innovative system consumes up to 40% less energy than a comparable air-cooled computer. And the CO2 balance is impressive: By directly using waste heat, the system reduces net emissions by up to 85%.

This is made possible by an innovative cooling system and powerful microchannel coolers attached to the back of the processors. Computer chips, which develop ten times more heat per square centimeter than a hotplate, can thus be cooled efficiently even with 60°C hot water. The entire cooling system of the computer is a closed, hermetically sealed circuit, enabling valuable waste heat to be recovered. Up to 80% of that waste heat can be repurposed by a heat exchanger that delivers it to a second, external heat cycle. In the case of Aquasar at ETH Zurich, the waste heat is fed into the campus heating system. Dr. Ingmar Meijer, Aquasar project manager at IBM Research - Zurich, explains: "With Aquasar we reached an important milestone in the development of low-energy and CO2-neutral data centers. This sends an important signal to the industry."

The next hot-water-cooled IBM system is already on the drawing board, this time in Germany. It will be significantly larger than Aquasar and is expected to go into operation at the Leibniz Supercomputing Centre (LRZ) in Munich, Germany, by 2012. Called SuperMUC, this new computer will be part of the Partnership for Advanced Computing in Europe (PRACE) HPC infrastructure and made available to scientists and research institutes throughout Europe. The system has a peak performance of 3 petaflop/s (1015 arithmetic operations per second) and is based on an IBM System X iDataPlex, which contains more than 14,000 Intel Xeon next-generation processors. SuperMUC will be more powerful than 110,000 PCs, enabling LRZ scientists to verify theories, develop experiments and predict results to an unprecedented extent—all this, while still requiring massively less energy.

Dr. Michael Malms, Director of Open Systems and High Performance Computing at the IBM Research and Development Center in Böblingen, Germany, states: "With the installation of SuperMUC we are taking the first step of research towards a system that can be used equally by academia and businesses. It is energy- and cost-effective as well as flexible, which makes it useful for many applications."

Future innovation: 3D integration

Looking further ahead, so-called 3D chips promise even higher performance with lower energy consumption. Paving the way for exascale computers, IBM scientists are pursuing extensive research on 3D integration. 3D chip architectures, in which processors are stacked on top of each other, not only reduce the surface area of the chip but also shorten the communication distance between the chips and increase the bandwidth for data transmission on the chip many times.

One of the main limitations in developing 3D chip layouts currently lies in the performance of conventional coolers. More complex designs with extremely thin, stacked processors can reach power densities of up to 5 kW/cm3 (kilowatts per cubic centimeter)—a power density which exceeds that of any current heat engine, such as internal combustion engines, by ten times.

At IBM Research – Zurich, novel concepts to scale cooling technologies for 3D chip stacks are being explored. In test systems, water is piped directly between the individual chip layers through microscopic channels measuring only about 50 microns. Such designs allow 3D stacks of heating elements to be cooled very efficiently with the heat fluxes released by today's processors.

Before first fully functional prototypes can be realized, which is expected to happen in the next seven to ten years, researchers must still overcome several technical hurdles. Their aim is to develop a system with an optimized flow of water through the thin layers, which at the same time reliably isolates the electronics from the water. A special difficulty is posed by the thousands of electronic connections that run vertically through the chip stack. In fact, the density of the components in such a system would be comparable to that of the human brain, which is intersected by millions of nerve fibers for signal processing, and features tens of thousands of blood capillaries for nutrients and heat transfer—without interfering with each other.

The three-dimensional integration of computer chips is one of the most promising approaches to boost performance tremendously while reducing energy consumption considerably. Supercomputers as small as sugar cubes could thus become reality.

Explore further: IBM Hot Water-Cooled Supercomputer Goes Live at ETH Zurich

Related Stories

IBM Hot Water-Cooled Supercomputer Goes Live at ETH Zurich

July 2, 2010

( -- IBM has delivered a first-of-a-kind hot water-cooled supercomputer to the Swiss Federal Institute of Technology Zurich (ETH Zurich), marking a new era in energy-aware computing.  The innovative system, dubbed ...

Sugar cube size supercomputers

November 15, 2010

( -- IBM labs in Zurich may very well shrink a supercomputer processor down to the size of a sugar cube making it almost 50% more energy-efficient than the world's leading supercomputers.

Recommended for you

Enhancing solar power with diatoms

October 20, 2017

Diatoms, a kind of algae that reproduces prodigiously, have been called "the jewels of the sea" for their ability to manipulate light. Now, researchers hope to harness that property to boost solar technology.


Adjust slider to filter visible comments by rank

Display comments: newest first

not rated yet Mar 02, 2011
The integrated circuit S-curve still promises ten to fifteen years of accelerating growth and we are already well on our way to creating its descendants.
not rated yet Mar 02, 2011
This old news, but this articles really explained it well this time. Great news nontheless. Kudos for IBM on more efficient use of computers.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.