Many people will be sceptical about introducing water cooling into their datacentres, especially at chip level. But this is exactly what IBM's latest datacentre model is based on, and it has strong total cost of ownership and renewable energy arguments on its side.
The IBM research project originates in its Zurich labs, and uses what is termed high-performance liquid cooling to channel water at high speeds through tiny copper pipes over the processors themselves.
The heat from the water is used to heat buildings, or for other civic purposes; for example, IBM is working on a desalination project in Egypt linked to a liquid-cooled datacentre.
IBM is currently building a supercomputer, Aquasar, based on the cooling technology, which reuses the waste heat carried by the liquid. Aquasar is a joint project with the Swiss Federal Institute of Technology (ETH), where it is located.
It is due to go live in April 2010, with the promise of reducing overall energy consumption by 40% compared with an equivalent energy-hungry air-cooled system.
Aquasar uses two blade servers that house specially adapted chips: a mixture of QS22 and HS22 IBM PowerXCell 8i processors, and Intel Nehalem Xeons. The water-cooled supercomputer will have a peak performance of about 10 teraflops when it becomes operational.
About 10 litres of water is needed to cool a single chip, pumped through a closed circuit at a flow rate of 30 litres a minute. This removes 85% of the heat load of the chips.
The processors constantly heat the water. This means it has to be cooled to the required temperature (approximately 60°C) by a passive heat exchanger. In the case of Aquasar, the excess 5°C of heat is removed and piped into the university's heating system.
The water pipelines from the individual blades link into a larger network, belonging to the server rack. This in turn is connected to the main water transportation network. The system is based on the human body, which pumps large volumes of blood through tiny capillaries using branch networks to reduce the pressure.
IBM said that water as a coolant has the ability to capture heat about 4,000 times more efficiently than air, and its heat-transporting properties are also far superior.
Dimos Poulikakos, head of the Laboratory of Thermodynamics in Emerging Technologies at ETH, said liquid cooling is the only way forward for datacentres. "We cannot afford any more to design computer systems based on the criterion of computational speed and performance alone. The new target must be high performance and low net power consumption supercomputers and datacentres. This means liquid cooling."
Bruno Michel, manager of advanced thermal packaging at IBM's Zurich Research Laboratory, added, "This project is a significant step towards energy-aware, emissions-free computing and datacentres.
"Heat is a valuable commodity that we rely on and pay dearly for in our everyday lives. If we capture and transport the waste heat from the active components in a computer system as efficiently as possible, we can reuse it as a resource, thus saving energy and lowering carbon emissions."
Michel said IBM is talking to several companies about trialling water-cooled datacentres, though they will not reach the mainstream for another five years, or 10 to 15 years if they use several layers of cooling pipes at chip level, which is termed "3D interlaying".
He argued that there is no need to worry about water and electrical components being used side by side, because electrical signals are isolated from the water pipes. Michel also said that IBM has run water-cooled mainframes since the 1980s, and has now had experience in running them for an extended time.
In addition, Aquasar constantly monitors humidity and the internal volume of the liquid. "The system can detect if there is any leakage into the electrical components. We are using all sorts of measures to avoid water leakage, but the system will shut down if it detects any," said Michel.
One issue that may put companies off adopting water-cooled datacentres is the initial hardware investment cost, which is between 10% and 30% more than for conventional datacentres. IBM's model requires the appropriate copper pipe-based cooling and heat distribution networks to be in place.
However, the benefits of going green are that computer room air conditioners and chillers are no longer required in the datacentre, which will slash energy costs by up to 50% and dramatically lower the facility's carbon footprint.
"If you can reduce energy costs, and also sell the heat, then the overall total cost of ownership is lower than for an air-cooled datacentre," said Michel.
- Youtube video on zero-emission datacentre
- T. Brunschwiler, B. Smith, E. Ruetsche, and B. Michel, Toward zero-emission datacenters through direct reuse of thermal energy
- H. Engelstaedter, Smart Energy Strategies: Meeting the Climate Change Challenge
- David JC MacKay, Sustainable Energy - without the hot air
This was first published in February 2010