War in the coding world

Author: Adrian Jackson
Posted: 11 Sep 2015 | 13:41

It's not often that the internecine rivalries of the HPC research and development community spill over into the public arena. However, a video recently posted on YouTube (and the associated comments), ostensibly a light-hearted advert for a SC15 tutorial on heterogenous programming, shows how real and deep these rivalries can be.

Update: Since writing this blog post the video has been taken down from YouTube by the person who posted it, it's not clear why, although the comments did get quite heated so maybe it just proved to be too controversial.  I have heard that the creators are hoping to put it back up again so if that happens I'll update this post with the new version.

The video is effectively a critique of OpenACC and CUDA by members of the OpenMP and OpenCL communities (the video also mentions MPI, but we can probably leave MPI out of this as it can easily be utilised by programs that use any of the 4 programming models previously menitioned). This type of discussion really does highlight that the technology for efficiently using HPC hardware is still very much a research area. Furthermore, it matters more than you may at first think, as many millions of pounds of money is invested each year in code development for HPC machines, and the machines themselves, just in the academic community. Ensuring that money is well spent on codes that can be easily used, development/maintained, and are performant across a range of HPC resource is essential in the current financially straitened times.

I should note that EPCC has a foot in both camps here. We are members of both the OpenMP and OpenACC standards organisations, and have staff who actively use all the alternatives for exploiting HPC accelerators and shared memory architectures. Also, I can see there are strengths and weakness in each of them.

I'm not going to say which I think is the best approach, you can view the video and read the comments, visit the standards pages and see what you think. However, I think it's very healthy that we do have alternatives. Whilst it may mean, in the long run, some programs need re-writing from one language or feature to another, it does mean that there are a range of approaches available which widens the possibities of what can be done both for application developers and hardware designers.  A single solution may not be able to support all hardware possibilities, or may not map well to all type of programs.

Which programming language or feature you use to parallelise your program, and exploit accelerators like GPUs and Intel's Xeon Phi, will depend on many factors: what is currently in the code, what platforms you have access to, what programming language you're using, and often what languages/features you know and what others are doing around you. However, it is only one of the ways that affects the viability of the investment of time (and therefore money) in an application or code. Documentation, program structure, tests, readability of source code, and many other factors can be equally as important, or more so, when it comes to maintaining, extending, and supporting an application. 

In computational simulation (the domain I work in) these application can be very large and complex, and last for many decades. The task of maintaining and extending such an application is not straightforward, especially when they also need to be efficiently implemented and able to exploit parallel computing. This, I think, means that rather than worry about the programning languages or features that are available for specific HPC hardware, it's much more important to design and construct your program in such a way as to be able to easily change the parallelism, or language feature, used without having to re-write the whole program.

Designing or refactoring a computational simulation application so the parallelism is distinct from the core computational kernels will enable investigation of OpenMP, MPI, CUDA, OpenCL, OpenACC, XMP, and many other parallel programming opportunities. Indeed, with optimal programming design it should be straightforward to convert between different parallel languages or hardware with little or no work from the developer.

So, whilst it is important the we have open and available standards and implementations for parallel computing, I think that better programming and software engineering skills for the application developers would give a bigger benefit for the money invested in computational simulation software. 

If you're interested in the skills required for writing and designing good programs, you could start by looking at the Software Sustainability Institute webpages.  

Author

Adrian Jackson, EPCC 

Comments

The video was removed by end user from youtube,

Indeed, it was removed not long after I wrote this. If it's re-posted (which I'm told it might be) I'll re-link here.

Thanks for this level-headed discussion about having multiple approaches to choose from. We need more healthy debates and fewer religious arguments on this topic.

Beautifully written, Adrian. Thank you. I completely second your thoughts that to a question of 'what programming model you will use', the most sane answer is 'it depends' and it depends on the factors that you listed and am sure the list goes on. Constructing and maintaining a single code base (at least minimal effort on reconstruction) for multiple architectures is an art; a real challenge so it is certainly an area to spend 'quality' time on.