Better programming languages for HPC

Author: Stephen Booth
Posted: 5 May 2016 | 16:43

Recently I seem to have had many conversations about programming languages for HPC. In some ways this is not a new subject - I have been having similar conversations for the last 20 years. However as HPC hardware evolves, machines become more complex and the issues that need to be addressed by programmers also become more complex. So it is not surprising that we are wondering if there is more the compiler could be doing to help us.

Trying to get a new HPC language off the ground is an almost impossible task. There is a huge investment in our existing languages both in terms of line of code and mind-share. Adopting a new programming language is also a very high risk strategy for a developer. It is interesting to note that OpenMP (one of the few recent developments that has obtained widespread adoption) is a directive based language that augments existing approaches rather than attempting to replace them.

We also should not underestimate the strengths of the languages that we currently use. It is very easy to dismiss "old" languages in favour of the new but this runs the risk of losing all the features and benefits that made the old language a success in the first place. The current dominant HPC languages (Fortran and C/C++) are very powerful and give the programmer a very high degree of control.

The problem, in my opinion, is not a lack of capability but a lack of separation of concerns. For example all of our current languages provide a great deal of control over the layout of data in memory. However it is not possible to change this layout without making a very large number of co-incidental changes. For example permuting the index order of arrays or converting arrays of structures into structures containing arrays. All of these changes are time-consuming and error prone when all the programmer actually wanted was the same program as before but with a different layout of data in memory.

Ideally we want simple extensions to our existing languages that allow us to make these kinds of changes without all of the associated side effects.

Interestingly the implementation of arrays in modern versions of Fortran are remarkably more flexible than you might think. Fortran allows you to refer to non-contiguous sub-sets of arrays as array-sections. As long as the programmer avoids some of the legacy features of the language it is possible for compilers to pass array sections as subroutine arguments without copying the data. Instead the array section is passed as an array-descriptor consisting of a base address and a set of stride values. It therefore seems reasonable to imagine a sub-set of the Fortran language where directives can be used to override the default row-major data layout of arrays without having to actually permute indexes throughout the code.

The HPF language also shows us that it is possible to construct extensions of Fortran where arrays become distributed data-structures where the concerns of data decomposition are handled by directives. Of course HPF no longer has any significant market share. The reasons for this failure are still open to some debate but I feel the lack of open source portable HPF compilers was significant.

The situation with C++ is also interesting. It is possible to use Object Orientation and C++ templates to encapsulate issues like data-layout and even decomposition. Many significant HPC codes written in C++ use these kinds of techniques within the standard syntax of the language. However this kind of encapsulation has to be designed into the application from the very beginning, it is not something you can add incrementally to an existing code.

It may be that the current capabilities of C++ are currently good enough, and the time is not right for proposing extensions to the language. On the other hand some kind of array-object extension which the compiler could map to either array-of-struct or struct-of-array memory layouts could be a very useful extension.

You can also read this post on our Medium(link is external) account.

Author

Stephen Booth, EPCC 

Comments

Stephen,

Have you seen this work, Arrow Street [1]? It might provide the capabilities that you are looking for. It provides a no-overhead way to write a single algorithm in C++11 to handle both AoS and SoA implementations transparently.

[1] https://github.com/ExaScience/arrow-street

Stephen Booth's picture

Yes you can do a lot with templates and this kind of thing is what I was thinking of when I said that maybe the current capabilities of C++ are maybe good enough. However incredibly impressive as this kind of thing is I don't feel the end result is as clean separation of concerns as could be possible. On the other hand this approach is pretty good and achievable within the current language definition.