Making the most of ARCHER for Materials Chemistry

Author: Iain Bethune
Posted: 15 Feb 2016 | 15:38

Ab initio modelling of oil formation in clay mineralsIn early December we added a visualisation of the most heavily used application codes to the ARCHER website.  At the moment it only shows data for the current month, but we've been recording the data since the ARCHER service began back in 2013 (table below).  It's perhaps surprising to see that, in a service with such a diverse user-base as ARCHER (the whole of EPSRC and NERC), 6 of the top 10 codes are materials modelling (ab initio or classical) and the top 10 codes represent nearly two-thirds of all the CPU time used on ARCHER!

Image: Ab initio modelling of oil formation in clay minerals (from UKCP http://dx.doi.org/10.1021/jp9096869)

Two of the major consortia of users on ARCHER are the UK Materials Chemistry Consortium (MCC) and the UK Car-Parrinello Consortium (UKCP) so it's no surprise to see that several 'ab initio' electronic structure codes - programs that calculate materials properties based on Quantum Mechanics - appear near the top of the list.  These codes can be very powerful and accurate, but are very computationally intensive, so getting the most out of them in order to deliver the most science possible within a fixed CPU budget is a top priority for these consortia.

Code CPU time
(node-hours)
% Time Users Area
VASP 8,576,269 16.33 262 ab initio
Unified Model 6,482,787 12.34 248 weather / climate
CP2K 3,751,406 7.14 134 ab initio / classical
Gromacs 3,208,405 6.11 121 classical
CASTEP 2,973,735 5.66 158 ab initio
HiPSTAR 2,034,182 3.87 13 fluid dynamics
LAMMPS 1,747,698 3.33 79 classical
ONETEP 1,691,174 3.22 39 ab initio
WRF 1,678,608 3.20 28 weather / climate
NEMO 1,447,704 2.76 34 ocean modelling

Table: Most popular codes on ARCHER, sorted by total CPU time.

To help address that very issue, I co-organised a workshop with STFC Daresbury's Scientific Computing Department (thanks also to MCC, UKCP and PRACE for funding!) to offer advanced training on both ab initio codes and methods, as well as using ARCHER to the max!  So for 3 days in mid-January, over 40 ARCHER users from around the UK as well as representatives from the top 6 ab initio codes - CASTEP, CP2K, CRYSTAL,ONETEP, Quantum ESPRESSO and VASP - descended on Daresbury for an ARCHER workshop.  

In order to be fair to all of the code developers present, each of them had a chance to give an overview of what their code does, highlighting areas where they are particularly strong, an 'Advanced Topics' session on a particular feature, and presented their roadmap for future development.  I also discussed how to make the most of some of the less-known features of ARCHER such as the large memory nodes, the Research Data Facility and Data Analytics Cluster, and of course how to measure and improve the performance of your calculations.  

Being able to tailor the content of ARCHER training for a particular user group is one of the benefits of having ARCHER 'consortium contacts' - members of the ARCHER CSE team who liaise with particular consortia, and have specific understanding of the type of work they are doing on ARCHER. In fact in this workshop I was wearing three hats: as a workshop organiser, the MCC's consortium contact, and as a developer of CP2K. It was a busy few days!

Image: Getting stuck into the tutorial sessions!

About half the workshop was left open for hands-on tutorial sessions - each code provided a set of exercises which could help either new users get to grips with that particular code, or to help advanced users learn new functionality.  Anecdotally, one of the major factors in choosing a code (or codes) for a particular research project is the researcher's level of experience with the code, or knowing a guru who can help them out, rather than if it happens to be best suited to the task.  

One of the aims of the workshop was to expose users to codes they might not be familar with and give them the chance to learn to use them - and many people did this.  But according to the participants' feedback, the best aspect of the workshop was getting to spend time with the code developers themselves.  Lots of problems were solved in a short space of time, and the only major complaint was that we didn't have more time to spend in the hands-on sessions.

On the final morning we had two plenary talks from Prof. Richard Catlow (MCC) and Prof. Stewart Clark (UKCP) showcasing some of the underlying theoretical developments and novel applications that give the UK a reputation for excellence in this area.  Both of them commented that this kind of workshop has clear benefits for their consortia, and recommended running it again every few years!