Uncovering the hidden costs of data centre cooling

STULZ UK’s Johnathan Attwood warns that environmental regulation is increasing the cost of conventional cooling approaches widely used by the data centre sector. How can operators reduce their OPEX and improve their green credentials?

There is a need for increased education in the data centre sector on the different cooling options available and their implications for operating expenditure (OPEX), energy efficiency, and environmental impact. Many data centre operators are familiar with Direct Expansion (DX) systems or chilled water type systems, but there is a need for greater awareness of new approaches that can help data centres deliver on long-term sustainability goals.

Cooling technology is advancing at a pace with the emergence of new technologies such as hybrid cooling, which has the potential to significantly reduce energy consumption and the environmental impact of data centres. With cooling accounting for up to 40 percent of the energy used in data centres, and one of the biggest outlays for operators, there is a need to understand the hidden costs associated with the various technologies and how they compare.

DX cooling

During the past 15-20 years, DX has become the dominant cooling solution and, until recently, this was driven by the fact that it was considered cheap and simple to install. DX units offer a number of benefits, such as good levels of cooling, coupled with a low footprint. As these systems are based on indirect cooling, there is also no danger of introducing contaminants from outside into the data centre.

However, DX systems are less desirable from an energy efficiency perspective as well as their dramatic instability, in terms of refrigerant pricing, meaning that these units are no longer the low-cost option that they once were.

Chilled water cooling

Chilled water systems have also come to the fore in recent years, as the capacities in data centres have steadily increased. However, hybrid solutions offer the advantage of having a lower refrigerant charge per unit – despite the fact that these operate using a similar process to a chiller. Some of the largest hybrid solutions, for example, use 7.6kg of refrigerant per circuit (with a twin circuit).

An equivalent, conventional chiller system will use around 40 percent more refrigerant on a 1:1 basis. On large systems it is also important to note that, if a chiller fails, there is a greater risk of losing a significant amount of refrigerant. With a hybrid unit, the capacity is spread over multiple units, but the risk of losing large quantities of refrigerant is significantly reduced.

Hybrid cooling

Hybrid cooling combines the reliability and control of a DX system, with the energy saving benefits of a free cooling system. In warmer months, when the external ambient temperature is above 20°C, hybrid systems can operates as a water-cooled DX system, where the refrigeration compressor rejects heat into the water circuit via a plate heat exchange (PHX) condenser. The water is pumped to an air blast cooler where it is cooled, and the heat rejected into the air.

In cooler months, below 20°C external ambient temperature, some systems cab automatically switches to a partial free cooling mode. In this mode, the water is directed through both proportionally controlled valves and enables proportional free cooling and water-cooled DX cooling to work together, with the dry cooler fans being used to cool the water to the desired level to achieve the required cooling capacity.

Mix mode cooling

Mix mode cooling is the point between free cooling and DX. This means the ambient air outside is cold enough to pre-cool the water provided to the hybrid unit but not quite cold enough to lower the temperature fully, to the required parameters. (Typically this is 3-6 degrees below the internal setpoint – dependant on system efficiency.) This ‘sweet spot’ requires an element of mechanical cooling as a ‘top up’ to meet the load within a room. This mix mode cooling typically makes up between 50-68 percent of the potential operation mode of a system.

Data centre operators can extend the free cooling and mix mode potential of their hybrid equipment by raising the return air temperatures. Potentially, it is possible to increase the return air from 24 degrees to 27 degrees, resulting in additional cost savings of up to 40 percent per year.

This hybrid approach drastically reduces power consumption over traditional DX systems and enables data centres to achieve huge cost savings. Data centre operators are also starting to scrutinise the pay back on their investments and this is where hybrid approaches can offer significant benefits. Payback depends on the size of the system and location, but the return on investment can be very quick, at around 3-4 years for a medium sized data centre (circa 500kW). The larger hybrid systems offer the quickest payback periods, however.

The hidden cost of refrigerant

The initial CAPEX cost for a hybrid cooling system is higher than a traditional computer room air conditioning (CRAC) unit. But there are other factors that need to be considered, when calculating the total cost of ownership and OPEX. As part of the EU Energy Strategy, there is a need to reduce fluorinated greenhouse gas (F gas) by -60 percent by 2030.

The latest regulation, which came into force in 2015, means that refrigerant availability and cost are being heavily affected with a revised target due in 2020 and yet further rises in refrigerant costs expected. As refrigerant is now a very expensive ingredient, this can have a significant impact on the total cost of ownership of the cooling technology.

For example, in August 2017 the average purchase price of R407c and R410a refrigerants was £14.78 and £29.78 respectively per kilo. By the end of July 2018, the average purchase price of R407c and R410a both increased to £50.66 and £58.59 respectively per kilo, and further increases followed shortly after. If we look at a 50m (equivalent length) and a total heat rejection capacity of 80kW, the cost of refrigerant equates to a cost of just over £3,600 per system, rising from £1,800 – representing an increase of 98 percent in just one year.

It is not just the hidden cost of refrigerant that poses a potential burden to data centres, however. In the future, there will be an intensification of certification for operating companies and staff, as well as a need for record keeping. Records will need to be kept for a minimum of five years and must include types of F-Gas used, quantity and how it is recycled when removed. It is therefore important to think about using sustainable solutions, today, and to reduce the use of refrigerants where possible.

Hybrid solutions require significantly fewer quantities of refrigerant and against this backdrop of rocketing refrigerant prices, are a significant advantage.

New low GWP refrigerants

There has been significant interest in the potential for a new variety of refrigerants with low global warming potential (GWP). The new ‘super refrigerants’ currently being reviewed by the data centre cooling industry are R1234ze and R1234yf. Unfortunately, the development of a low GWP refrigerant CRAC DX solution has been slow to come to market and there are trade-offs that will need to be considered in the future.

It is likely that the initial cost of low GWP refrigerants will be higher and flammability will also require careful risk assessment – butane, for example, is a good refrigerant, but is incredibly dangerous. New refrigerants will face similar issues and will come under increasing scrutiny. A large, single chiller using 200kg of refrigerant presents a much greater risk of leaks, than when using a hybrid system, comprising multiple small units – each using 18kg of refrigerant. Data centre operators are going to become increasingly aware of this issue as regulations around refrigerants continue to be strengthened.

Sustainable future

Next Generation Data (NGD) and other high-profile data centres are now adopting hybrid technology, and data centre managers are finding that they can save costs, maintain their green credentials and ensure a high level of resilience. As the cost of refrigerant continues to rise, conventional technologies will become increasingly expensive to operate. In the future, hybrid cooling could help reduce the risks associated with refrigerants, while taking advantage of sustainable free cooling approaches.