Cooling the Internet Panel – How does the Data Centre Industry Keep Servers Cool Without Warming the Planet
Long read
On 1st February 2022, Airedale hosted a panel discussion featuring industry-respected contributors from both end user organisation and engineering contractors, to discuss the burning topic of sustainability with data centres. Broadcast on the DCD platform, and available to watch here, this blog summarises the discussion.
The panel consists of the following people:
- Abed Jishi – VP Design, Vantage
- Andy Hayes – CTO, Yondr
- Dan Loosemore – mediator, DCD
- Daniel Burczyk – Business Development Manager, Data Centres, Airedale
- Jamie Hughes – Head of Technical, Data Centres, Bouygues
- Paul Pompili – Divisional Director, ISG
The panel discussion opened with an introductory question looking at what approach to innovation the data centre industry needs to take today, with new challenges ever-presenting themselves. It was acknowledged that there has been a positive movement towards sustainability, with a focus on PUE and power usage for increased efficiency.
Opening the discussion was Abed Jishi from Vantage, making the point that the starting point is to adapt sustainability as a mindset, starting with planning and continuing this mindset through to design and seeing sustainability as a great driver for innovation.
Andy Hayes of Yondr agreed with this point and noted that sustainability is key to the future of innovation, adding that it is essential to work with clients to understand the future of technology and technology change. Looking 10-15 years in the future to what the future of IT may look like, and planning in flexibility to accommodate future IT change is crucial, as well as being mindful of energy efficiency and sustainability whilst on that journey.
Daniel Burczyk of Airedale commented on how the data centre industry is under the spotlight and there is more awareness than ever of the energy consumption of a data centre, making everyone aware that things need to change and the industry as a collective needs to act. Daniel explained that at Airedale, the key is to understand the value chain of the data centre and contribute to a point where we can build more sustainable data centre, beyond product level. He emphasised the importance of building relationships with everyone in the value chain to allow us to achieve targets set by both government and ourselves.
The key is to understand the value chain of the data centre and contribute to a point where we can build more sustainable data centre, beyond product level.
Jamie Hughes from Bouygues referred to Daniel’s point about early collaboration, not just around product selection but also right back at the planning stage, highlighting the implication and importance of design and build of a data centre. Jamie felt the challenges very much vary depending on the location of the site, mentioning how geographically the cooling technology can change depending on the ambient environment, and also the pressures in terms of things like energy resource and control of emissions are also different and as such there is no silver bullet in terms of cooling solution. He commented on how the source of the challenges has changed – 10 years ago it was predominantly speed and scale, but now it comes in a different form as pressure from society, government and local authorities and this means contractors and end users should be mindful of how things are measured, to ensure they are delivering real results.
He finished by saying how operators, designers and builders all play an important part in finding the right solution for the client and end user requirements, that meets the pressures of both society and themselves to make things cleaner.
Paul Pompili of ISG then took this point of collaboration one step further, observing how clients are looking for the contractor to take ownership of the design, but unfortunately the contractor gets involved in design development far too late and as such opportunities to make greater efficiencies are missed. Paul described design teams that work with a silo approach, whereby the architect and structural engineers work quite closely together, but the M&E consultants have less of a team effort, looking at architect to solve and wrap their solutions around them, and Paul put forward the argument for a more unified approach.
He also raised the point that there is a lot of discussion around heat generation within the data centre and the mechanical solution to provide the cooling. He suggested some of this heat generation could be designed out of the building at a much earlier stage and commented how he would like to see more innovation; looking at more intelligent fabrics and looking at design of the building itself to solve some of the problems and reduce the amount of energy put in to these buildings.
Paul concluded by he would like to see a more unified approach from the start of a project, utilising everyone’s knowledge to achieve a better solution. Making the point that chip technology isn’t improving in terms of efficiency, a different viewpoint is required that focuses on lower energy solutions. He acknowledged there might be a trade-off between the space of the development and energy used in the building to allow for increased energy efficiencies, but that these sort of things need to be looked at in the early stages with the developer and the customer and thus it requires people working as a team instead of individual engineers.
Conversation then moved on, with Dan raising the issue of how an end user might evaluate a site for power, mechanical and electrical requirements, and how would they evaluate greenfield projects. Abed noted that whilst most operators strive for scalable, reliable and efficient designs, incorporating sustainability early on whilst gaining stakeholder engagement is essential. Abed stated that a replicable design is required, which then affords opportunities to be more efficient. By reducing complexities and bringing in local expertise, sustainable, replicable and efficiency can be created through the process and design-led thinking allows teams to achieve a prototype creation that can improve efficiencies.
Whilst most operators strive for scalable, reliable and efficient designs, incorporating sustainability early on whilst gaining stakeholder engagement is essential.
Dan then raised a viewer’s question that had been submitted, asking what the panel felt about embedded carbon in construction and waste at the design build phase?
Andy, agreeing with Abed’s earlier point, commented that standardisation and continued evolvement and improvement is a good way to become more sustainable and efficient and said how Yondr are now looking at the full life cycle cost of facilities and request EPDS from suppliers, to allow them to consider the carbon cost of products. Andy noted that the data centre industry can learn about the benefits of standardisation from other industries such as manufacturing, to improve quality and sustainability and ultimately achieving greater efficiency by being more standardised. He mentioned the benefits of manufacturing off site, to reduce the required skill set of labour on site and improve things like delivery cost and time, and improve sustainability; an area of discussion he goes more deeply into later on. Andy also supported Paul’s earlier point of getting the right people involved earlier in the design process, to reap the benefits of that later.
Daniel supported all of the points made and commented on how Airedale maintains a keen eye on how they can contribute to the sustainable data centre. Daniel’s concern is that whilst PUEs are driving metrics and the industry has experienced improvements over the past 10 years, these improvements are incremental and the next thing to address is what metrics should and could be used to assess the data centre of today. Alongside PUE, water use effectiveness and embodied carbon offer valuable environmental insight, and we perhaps now need to widen the lens to ensure we don’t sacrifice one section of a data centre to another. With solutions getting close to a PUE of 1.0, the question is what does the sustainable data centre of the future look like? And can other technologies use the energy we are creating?
With solutions getting close to a PUE of 1.0, the question is what does the sustainable data centre of the future look like?
Dan then introduced another statement from the viewers’ questions, that data centres demand revolution rather than evolution and one viewer asked what stands in the way of embracing new technology, with Dan reflecting on the way mission critical data centres prefer to adopt tried and tested technology.
Andy observed that whilst there are emerging technologies in the market such as direct-on-chip, liquid cooling or immersion cooling, they are not being adopted at scale and the solution employed to cool a data centre is very much the same as it was 20 years ago, with air passing through a server, being heated up, going back to a cooling system, being rejected and going round the cycle again. That said, Andy suggested that a new technological change gives opportunities to do something differently within the space, for example immersion cooling gives much higher return temperatures back into the water system which is more useful for reuse of heat. He noted that with greater efficiencies, compressors and chillers won’t be required to back things up, and this is where things can change in time, but at the moment demand at scale is based on air for a form of cooling.
Andy then considered the impact of the SLA and noted that recent times have seen temperature ranges opening up. He explained that in some instances he has seen temperatures of up to 35 degree centigrade for periods of time, where as they used to be very fixed between 24 to 26 degrees centigrade, and that drive and change of SLA will help with some of the innovation and help providers hit sustainability and energy efficiency targets. Andy explained that everyone in the data centre industry works with SLAs and they impact things like redundancy, temperature and humidity as well as the way they operate and run the space, but they all differ. Some companies set their own internal SLA, other companies will be setting an SLA and passing it out to their customers and some have their customers impose an SLA upon them.
Building on this point, Dan asked if less energy consumption with higher temperatures is possible and what is best practice? Abed responded, stating he has seen a shift from operating within the ASHRAE recommended envelope, to the allowable range. He has seen supply temperatures increase from 27 degrees centigrade to almost 30 degrees centigrade, but said it was important to recognise that what an operator can control is quite limited. End users are driven by availability of equipment which are limited by temperatures and it is a cycle. He noted how Vantage has built facilities that can withstand 100% or even 110% of the load dictated by customers, but the ramp up to withstand the 100% sometimes takes a few years so agility is important in operations and it is critical to work smart.
End users are driven by availability of equipment which are limited by temperatures, and it is a cycle.
Abed again emphasised the point that having sustainability as a mindset, through design, operation and construction, means there are some things that can be controlled and improvements that can be incorporated within the white space. Abed gave the example of hot and cold aisle containment, describing it as a simple, cost effective mechanism that has significant impact. He then added that other technologies like sensors, AI and software can be employed to read how the heat load is being distributed and to apply the right amount of cooling where it is needed the most. All these initiatives support the view that agility of operations and being smart and efficient lead to more sustainable data centre environments.
Daniel noted again the developments of the past few years in relation to rising temperatures and the need to get to a point where the envelope can be opened up beyond incremental changes, but this would require new technologies, which would need to be proven before being rolled out into a 1MW facility. He commented on how expansions over the past 3 years have been fast and the message he gets is not that the data centre provider is chasing the latest technology, but instead demands speed of build; Daniel believes the more sustainable data centre has to develop alongside market growth itself. Adopting technology and with more companies opening up the envelope – which we give recommendations on – is leading to change. It may be happening slowly, but it is happening.
The next question Dan proposed was how does increased demand impact selection criteria and how do end users prioritise solutions?
Abed was very clear that customers seek confidence in design and for that to happen, Abed has to have confidence in his equipment. He said one of the drivers for sustainability is having something that can be replicated, and as such it is important to have confidence in the performance of equipment in different climates and locations, to deliver different capacities. Confidence and performance are in his view key for selection process.
One of the drivers for sustainability is having something that can be replicated, and as such it is important to have confidence in the performance of equipment in different climates and locations, to deliver different capacities.
Andy seconded this view, stating his customers also demand global consistency, which in itself leads to standardisation., He noted that a facility’s life cycle could be up to 20 years, whereas IT refresh can be 3 or 4 years and as such there is an imbalance, which they address by building flexibility into their design. He acknowledged that new technologies like immersion cooling are being explored, but is uncertain as to whether they will ever be deployed at hyperscale level, noting it will very much depend on how reliable they are in comparison to the current way of working. He finished by stating the future is probably a very different technology altogether, but we are a while off that and so for now, having the flexibility to adopt new emerging technology over the next 10 to 15 years is key.
Jamie concurred with Andy, that there is a consistent air cooled standardised IT model in operation at the moment, which may vary within the detail, but is ultimately the same cooling solution. Jamie said he has experienced developments with end users giving very clear detailed requirements about how they will deploy the initial step of IT and this brings opportunities of optimisation of the facility, stating that it is possible to have a standard model with the ability to optimise the solution to drive energy efficiency. Jamie noted that end users are giving the information much earlier.
It is possible to have a standard model with the ability to optimise the solution to drive energy efficiency.
With regards to selection criteria from a sustainability point, Jamie felt there is more than energy to be considered and commented how the changing use of refrigerants may have been driven by EU regulations, but has continued citing the shift from R410A to R32. Jamie stated there is a real tangible benefit from this step change, that wouldn’t be reflected in PUE. He also added the step changes around embodied carbon are happening and are promising.
The final question raised by Dan was that of circularity with the data centre industry, stating that PUEs doesn’t encapsulate everything and if the limit of our ambition is to heat up the sea more slowly then perhaps we haven’t done enough?
Paul opened the discussion around this point, noting that ISG are keen on heat reuse, but one of the challenges seen from client teams is that they understandably want to maximise their real estate and so restrict the data centre facility to smaller spaces, which then restricts the engineering efficiency solutions opportunities. Paul felt that if the data centre could be associated with horticultural environments and residential facilities, then perhaps a better method of reusing the heat could be developed, especially in the horticultural sector which could use it year round and would be beneficial to the local environment and local population as well. However he noted there are limitations due to the density of the buildings in city locations. He cited facilities in excess of 100MW in say a central London location, stating that if the capacity could be reduced to 40 or 50MW, then there would be more space which could be utilised for more environmental benefits. Paul believes contractors need to start challenging the mega users of hyperscale data centres, reminding them of their environmental responsibility and encouraging those in the commercial sector to reuse their energy effectively, as he has seen in some public funded projects. Paul gave an example of a project he worked in 2008, which was a large data centre for a government body. It was extremely large and less efficient in terms of its square meterage in todays terms, but it was very low energy with PUE 1.1, all free cooling and no chillers involved; it continues to provide a high density cooling facility for the government today. Paul noted that some of the biggest organisations in the world are making lots of revenue from using these data centres, but the benefits need to reach the environment and local population as well the shareholders.
Contractors need to start challenging the mega users of hyperscale data centres, reminding them of their environmental responsibility and encouraging those in the commercial sector to reuse their energy effectively.
Andy stated the data centre does need to be more useful and consider its impact is on the local community and environment. He explained a data centre with a standard air cooling solution kicks back 35 to 40 degrees centigrade water and he questioned how useful is that into legacy or existing heating systems, however it could possibly work with new build residential opportunities where a new district heating system could be designed to work with. That said, he explained that some district heating systems are tied into power stations, and a data centre couldn’t really improve on what that can offer. He explained that things like swimming pools and greenhouses offer opportunities, but cloud computing facilities are typically built in urban areas where there is little call for such things. He felt the most opportunities come from larger hyperscale data centres in big out-of-town facilities and he has seen examples of such work being done, along with examples of sea water cooling, but it is very much location dependent.
A further point Andy made, which isn’t location dependent, is the impact a data centre project can have on a local community. He explained that the development of a large scale data centre may bring 1,000 or even 2,000 people into an area and this helps them for the construction phase of maybe 2 to 3 years, but then they pull out, leaving just 20 or 30 highly skilled people who have been brought in from outside the area, thus the project having a very obvious positive, then negative impact. By considering the overall lifecycle of the data centre, the focus shifts to how to positively impact the area. Going back to Andy’s earlier point of having lower skilled people on site during construction, he explained that by delivering part-constructed elements, it is possible to train people in local area so they can build the centre. Further to this, embedding technology in the operation of the site such as AR and VR means less skilled people can work in the operating environment. These people can be trained within a few months, eliminating the need for lengthy experience and formal qualifications, and these people will stay and improve the community.
By considering the overall lifecycle of the data centre, the focus shifts to how to positively impact the area.
Abed was keen to clear up a misconception that the data centre industry is resisting supplying heat to its community, explaining that the investment required for any data centre to have a facility for heat reuse is a very minimal. The issue usually is the infrastructure required to take the waste heat away and the councils or local authorities don’t necessarily have the funding for it. Abed suggested a collaborative approach between local councils and communities to demonstrate how both entities can benefit from the infrastructure might be a solution. Abed finished by confirming all Vantage DC are heat reuse ready, as are many other providers, but they need the systems to connect to.
Daniel closed the discussion, observing that the conversations around collaboration within and beyond the data centre industry can lead the way to improved efficiencies with the facilities, noting that further sharing of information and increased integration between data centre providers and local authorities could lead to more environmental advancements within the community as a whole. He recognised some progress, in that the conversation looking at how to do this is at least happening now.
Daniel mentioned the intense focus on efficiency delivered by Airedale and how this is afforded by its specialist engineers and state-of- the-art research and development facilities at its UK HQ. He concluded that Airedale products facilitate the current market, making the required incremental changes but eventually we need to be ready to integrate the data centre into the community and be ready for what that might look like.