Advanced Computing Facility: EPCC’s secure, resilient data centre

EPCC is the UK’s most experienced provider of large-scale compute systems. In addition to national services, we operate a wide variety of smaller research systems. The Advanced Computing Facility (ACF) is one of Europe’s largest research computing data centres and is acknowledged by the AI and high performance computing community as a global leading supercomputing site.

 

30 years of excellence 

Over the past 30 years EPCC has operated all but one of the UK’s National HPC Services. At a national level, it has also operated HPC systems for the DiRAC consortium and the EPSRC Tier-2 programme. Since 2003, all its services have been operated from its Advanced Computing Facility (ACF) data centre on the outskirts of Edinburgh. 

Additionally, since 2018 EPCC has designed and operated the Edinburgh International Data Facility (EIDF) – a facility focussed on Data Science and AI - the key IT infrastructure for the Edinburgh and SE Scotland City Region Deal. This includes a growing Regional AI GPU Service which is currently composed of 500 GPUs. The AI services include a cluster of two Cerebras CS-2 AI supercomputers and a Graphcore Pod64 Bow IPU system. 

Investing in future skills 

We are committed to promoting and growing the skills required to develop and operate complex supercomputing and data science systems, and to help young people find their way into careers in our field. 

This includes an established internship programme with undergraduate students and Graduate Apprenticeships from Edinburgh Napier University. In summer 2024, we also employed a number of mechanical and electrical engineering students from the School of Engineering at the University of Edinburgh to gain an insight into working in a global leading supercomputing site. 

Minimising energy use 

Environmental sustainability improvement is a continuous activity at EPCC. Cooling to our systems uses a variety of methods including direct air cooling, semi-direct water cooling and direct water cooling. Free-cooling, where water is pumped to fans on the roof to be cooled solely by outside air, is used automatically when the outside temperature is lower than the temperature of the chilled water circuits (much of the year in Scotland!). We also have variable inlet and outlet water temperatures for our systems and rooms to maximise efficiency. 

A trusted partner 

All our services involve handling user data. We are responsible for ensuring the appropriate level of technical controls are applied to the data to ensure that only the appropriate people can access the data, the data is undamaged, and is available to data owners. 

Net Zero and innovation

All our electricity is certified by the Renewable Energy Guarantees of Origin (REGO) scheme, which demonstrates that electricity has been generated from renewable sources only. As a result, the ACF operations are formally categorised as Net Zero by the Scottish Government.

We are actively researching and developing a digital twin of our data centre to enable modelling and optimisation of our energy and resource usage. The digital twin will help us unlock more efficient system operations, model future systems, and design our machine rooms to maximise efficiency. 

The digital twin combines modelling and simulation of power and cooling hardware, along with knowledge of the applications that run on our HPC systems, to predict and explore the interaction between applications running on our systems and the hardware that supports them.

Other innovative projects relating to better use of our excess heat include reducing emissions by sending heat to a Combined Heat and Power plant for use by nearby University of Edinburgh buildings. 

We are also part of a £2.6m collaboration exploring the use of flooded mines around the ACF to store and distribute waste heat, with the ultimate aim of making this heat available to homes and businesses.

Technology

The ACF operates a 200 Gbit/s Data Centre Network to support systems hosted on-site and connects to JANET (the UK’s Joint Academic NETwork) at 100 Gbit/s for user access. 

As part of the longer-term strategy for the site, capacity has been upgraded from 8MW to 38MW.

Security

We have a commercial-grade data centre firewall from Checkpoint, which operates at 100Gbps.

EPCC holds the following certificates in the internationally recognised ISO standards:

  • ISO 27001: Information Security
  • ISO 9001: Quality Management
  • ISO 22301: Business Continuity and Disaster Recovery.

As part of the management of systems under these ISO processes, we run an active business continuity and disaster recovery (BCDR) process. We continuously test and evaluate our risk management and contingency plans, as well as our standard operating procedures, to ensure service delivery is undertaken in a secure and robust manner

We are accredited for provision of data by the UK Statistics Authority. 

Further information

To find out more about the Advanced Computing Facility, please contact us at: info@epcc.ed.ac.uk

Read about the Advanced Computing Facility on the EPCC website.