Worldwide co-design centres & software development for exascale
Posted: 24 Feb 2015 | 12:05
The European Exascale Software Initiative is a consortium of 29 organisations and around 100 individuals who are working towards providing key recommendations on European policy with a particular focus on how software can be developed and techniques further improved to help meet the challenges that exascale computing might bring.
The first version of this project, EESI-1, highlighted a number of key areas for further investigation and consideration. The EESI-2 project, which has been running since 2013, has built upon this and focused its attention on these areas.
EPCC has been working on one of the tasks of EESI-2, which is to identify the state of the art of worldwide co-design centres and come up with some concrete recommendations about how the current approach might be adopted and/or modified to assist with software development in a future exascale era.
Co-design itself involves multiple stakeholders, from different backgrounds, working together to produce an overall solution. The concept has actually been around for some forty years in other industries and it is believed that this methodology, where people with a wide variety of skills work together in a team to produce the end product, is likely to be a key enabler in the next generation of scientific discovery. EPCC was ideally placed to carry out this work as we are leading efforts in using co-design for exascale projects.
Whilst there is much literature out there about numerous co-design centres and the models that they have adopted, there is no substitute for talking to people in the community. In order to understanding how people view co-design, when they have used it, benefits or disadvantages of the approach and suggestions of how this might be best harnessed in the future, we initially developed a questionnaire. It was publicised in a variety of different manners such as at conferences, on email lists and also a previous blog post. We got a great response to this and, based upon the answers and themes identified, then targeted specific individuals to conduct in-depth one-to-one interviews with them.
There were many interesting points raised by our investigations; such as a wide variety of definitions about what co-design actually is and the importance of managing the staffing when it comes to different skill sets required by a project at different times. One of the recurring comments was that it can be difficult with people from different backgrounds to ensure that communication is effective and how to drive the entire team to a “common good” rather than people focussing on their own individual goals. The point was also made that education is a key enabler and even elementary courses and workshops in a specific field can be very helpful to those with expertise in different areas. Many mentioned that building effective co-design teams is all about breaking down barriers to collaboration. It was evident that there is some confusion between a Centre of Excellence (CoE) and co-design centres, everyone agreed that these terms are used to describe very similar approaches but some considered them to be exactly the same, whilst others highlighted some subtle differences.
A number of existing organisations and projects have adopted co-design as a way of working. One of the more visible efforts have been the three US Department of Energy (DoE) centres which address the hardware, software, numerical methods, algorithms, and applications in a specific area that is seen as a grand challenge for the DoE. These have been very successful and since they were set up have produced significant science.
Co-design has not only been adopted in the US - a number of European projects have used this methodology too. One recent example of this is CRESTA which, led by EPCC, brought together leading supercomputing centres, equipment vendors, programming tools providers and six application and problem owners to explore how the exascale challenge could be met. It was very much application focused with a major aim being that their work would enable new science, which was previously unattainable, to be performed. The six applications that CRESTA focused on were known as co-design vehicles, and individuals with a wide variety of expertise have worked on them together to advance them towards the exascale future.
Based upon our own investigations, analysing existing co-design implementations and projects we have come up with a number of recommendations. These are available in more detail in the white paper and I just highlight a few here.
There are European calls for CoEs and co-design centres. Based upon this, workshops should be funded to bring communities together and support them in applying for funding. This is similar to how the US DoE set up ExaMath, a new co-design centre for applied mathematics. In order to reach exascale, and support science that takes full advantage of these future resources, the EU should expect to increase the budgets allocated to funding co-design centres. It should be appreciated that this is a worldwide effort, and the co-design activities in Europe should be organised in such a way that complements those which are currently happening in the US.
You can view the full deliverable, along with the detailed investigations and recommendations, on the EESI-2 website.