EuroCC and The UK National Competence Centre

The project finished at the end of December 2022 and has achieved much during this time. Both partners collaborated across a wide range of areas, with EPCC focusing its efforts on training, industry engagement, and facilitating scientific and technical expertise transfer.

Accessible training

EPCC has developed an exemplar course for accessible training. Given the aim to broaden the base of HPC users, we decided that the best topic would be a "Hands-On Introduction to HPC" course where key HPC material is explained and illustrated with practical examples to be run on a real HPC system. This builds on our experiences of developing the PRACE Supercomputing MOOC, which was aimed at a more basic level, designed for people with no previous computing experience and containing no hands-on exercises.

A subsequent technology survey was conducted to identify the best delivery platform for the new course. Our survey identified a system co-developed by NCC Sweden that was best suited for our purposes (see our article Developing online self-service HPC courses). The beta version of the course is now available and we are taking input and feedback on ways to improve it. Access the course here. 

Online training has proved to be both accessible and configurable and our ongoing HPC training activities are publicised via the EuroCC training portal. Over the course of the project, 258 users from 23 EuroCC member countries (excluding the UK) have attended EPCC’s live-delivery online training. 

Technology transfer

UK technology transfer was our second area of focus, and we have been working to ensure the correct mechanisms are in place to support it. During the course of EuroCC, EPCC has engaged with approximately 340 industry contacts. This includes meetings and phone calls, but does not include social media posts. EPCC has carried out over 30 high-performance computing and high-performance data analytics/artificial intelligence projects with industry in this period.

These projects have been carried out with both large and small companies, with an approximate split of 20% with large companies and 80% with small companies. The range of applications include energy, medicine, engineering, satellite imaging and optimisation. 

Two pilot project examples are:

• Danu Robotics working with EPCC to develop a system which aims to automate waste-sorting at recycling plants – also known as materials recovery facilities (MRFs) – with speed and accuracy, thereby increasing the quality of output and reducing costs. EPCC’s role has been to provide consultancy and support to Danu Robotics as it develops its machine-learning software. See article Revolutionising recycling with AI.

• EPCC has helped a UK start-up, Mercury Environmental Systems Ltd, through support and consultancy, to develop a service that uses satellite data and high-performance computing to monitor and forecast crop growth. This Mercury data service aims to aid more efficient and sustainable farming as well as broader ecological and environmental decision-making. Read more in this article: Data Slipstream: bringing together Earth-observation data, science, industry, and next-gen compute.

See also the article Technology transfer: EuroCC@UK success stories.

Facilitating scientific and technical expertise transfer

We carried out a series of technology state-of-the-art investigations, each of which produced a report on a topic of interest to HPC application developers. 

Rust for HPC applications

The first investigation looked at the use of the Rust programming language for HPC applications. Rust is a relatively new language: like C++ it is object-oriented, but it has in-built features that help enforce memory and thread safety. These are intended to pick up many otherwise difficult to detect bugs at compile time, without compromising on performance. The mini project considered ease of use, portability, programmability and performance (both serial and multithreaded), using a simple test case. The report for this will be published by the end of year 2. See our article Emerging Technologies: Rust in HPC.

Task-aware communication libraries

The second investigation focused on task-aware communication libraries. Programming using tasks with data dependencies is a powerful method for minimising idle time due to excessive synchronisation and load imbalance in parallel applications, and is supported by APIs such as OpenMP and OmpSs. Implementing a full tasking model efficiently on distributed memory is very challenging, so a practical compromise is to use a hybrid of tasks within a node and traditional communication library such as MPI between nodes. Barcelona Supercomputing Center (BSC) have developed task-aware versions of these libraries, and the mini project used them to implement some simple kernels and mini-applications, and assessed the performance, ease-of-use of, and available support for, this novel programming model. 

Increasing MPI-IO performance on ARCHER2

The third focused on parallel IO. File input and output can become bottlenecks for parallel programs running on large numbers of processors. In this investigation we looked at how to increase the MPI-IO performance on ARCHER2, the UK National HPC service, using the Cray Lustre lockahead options which have not been previously investigated on this system. We then investigated the performance of ADIOS2, a more modern parallel IO library, and look at the performance of IO from a real HPC application rather than a synthetic benchmark.

All these investigations can be found at:

https://www.archer2.ac.uk/research/technical-reports/

And finally...

The project finished at the end of December 2022 and while EuroCC2 continues, Brexit has unfortunately ruled the UK out as a participating partner. We do however hope to continue to collaborate with our European colleagues to help UK researchers benefit from best practice across Europe, so watch this space.