Unwrapping Tesseract: the new DiRAC system hosted by EPCC

DiRAC is part of the UK Government’s investment into the UK’s e-infrastructure to improve computing, software development, data storage, networking and distributed computing networks.

In January 2018, EPCC’s Advanced Computing Facility (ACF) retired the Science & Technology Facilities Council’s (STFC) DiRAC BlueGene/Q service. In March it was replaced by the STFC’s new Extreme Scaling system, an 844-node Hewlett Packard Enterprise 8600 “Hypercube” supercomputer.

Each node has two Xeon scalable processor 4116 “Silver” nodes, and 96 GB of memory. In total, the new system has 20256 Intel Skylake computing cores, and is interconnected using 100 Gbit/s Intel Omnipath interconnect.

The Hewlett Packard 8600 is based on the former SGI ICE-XA technology, with tightly coupled network and energy-efficient water cooled technology. Like its predecessor, the system is uniquely optimised for Cartesian PDE problems that arise in computational particle physics simulations.

Unlike other 8600 installations, the Edinburgh system uniquely makes use of an exact power of two (16) nodes on each leaf switch, assisting the science by placing the Cartesian problems precisely on the underlying hypercube network topology. The system’s name, Tesseract, highlights this unique use of topology and customisation to the target problems.

In addition to carrying out the critical infrastructure installation and system configuration work required to install and operate the system, the ACF and HPE teams have assisted STFC scientists in demonstrating that large computing jobs can run all the interconnect links concurrently at wire speed, making for a highly scalable simulation platform for the target class of problem.

Commercial on-demand access

Tesseract is available for commercial use. Contact Thomas Blyth, EPCC’s Commercial Manager, to find out more.

DiRAC Extreme Scaling

Read more about DiRAC at EPCC: hwww.epcc.ed.ac.uk/facilities/dirac