EPCC's Advanced Computing Facility: planning for future growth

Author: Paul Clark
Posted: 1 Feb 2021 | 12:10

Since 2005, the Advanced Computing Facility (ACF) has housed all the major systems managed by EPCC. It has expanded and evolved since its creation, becoming one of the most innovative and efficient facilities of its kind in the world.

The building and its internals have changed greatly since I started in February 2018, as part of a drive to ensure that our wider master planning for the site is reflected in what visitors see. This includes a video wall using Raspberry Pis and PiWall software to allow us demonstrate HPC visualisations to visitors.

We are developing a site-wide Data Centre Infrastructure Management (DCIM) approach which allows us to view real-time data or room and system performance on screens outside of different rooms and on our video wall.

In addition, the ACF has had significant investment over the years, most recently with the creation of Computer Room 4, the home of the new Edinburgh International Data Facility (EIDF). We also host and support a number of other HPC systems at the ACF, such as the National Tier-2 system, Cirrus. The first phase of the next UK national supercomputing service, ARCHER2, has also been installed.

Cooling systems

Each of the ACF’s four computer rooms hosts specific HPC equipment and is supported by associated plant rooms that provide dedicated power and cooling infrastructure. We have several pieces of equipment and racks at our site which are “traditionally” air cooled, but the majority of our equipment and speciality high-performance computers are water cooled. As water is a significantly more effective coolant than air alone, the ACF is an extremely efficient data centre.

Thanks to the Scottish climate, for much of the year the ACF benefits from free cooling. The water which supports our cooling infrastructure is pumped to our roof-mounted dry air cooling towers and back again, allowing the lower outside temperature to cool our water as it passes through the towers. It is only on extremely warm summer days that there is no free cooling and we then use our large-scale mechanical chillers to cool the water circuits.

Edinburgh International Data Facility

EPCC plays a major role in the Data-Driven Innovation (DDI) Programme of the Edinburgh and South East Scotland City Region Deal. At the heart of this programme is the EIDF, which will be housed in the ACF’s new Computer Room 4 (cr4).

This high-resiliency room provides the state-of-the-art computing and data-storage infrastructure needed to develop the technologies that will enable data-driven innovation. cr4’s cutting-edge networking advancements, which will underpin the EIDF, are among the fastest available while its innovative cooling solutions make it one of the most efficient computer rooms possible.

We have run the National Safe Haven for several years and we are building on this deep expertise to provide a secure data-hosting and analytics infrastructure for the EIDF service. EIDF will hold unique datasets and allow approved, authorised users to find innovative solutions to novel questions. cr4 will be capable of hosting up to 270 cabinets, giving EIDF and the DDI programme a platform on which to grow over the next 10 years.

Links

"EPCC’s Advanced Computing Facility: how it all began" blog post
www.epcc.ed.ac.uk/blog/2021/advanced-computing-facility-how-it-all-began

ARCHER Tier-2 National HPC Facility
www.archer2.ac.uk

Cirrus Tier-2 National HPC Facility
www.cirrus.ac.uk

Data-Driven Innovation Programme
ddi.ac.uk

Edinburgh International Data Facility
www.ed.ac.uk/edinburgh-international-data-facility

National Safe Haven
www.isdscotland.org/Products-and-Services/eDRIS/Use-of-the-National-Safe...

Author

Paul Clark, EPCC

Image: ACF plant room.

Blog Archive