Energy efficiency in parallel computing

Author: Michele Weiland
Posted: 14 Jan 2015 | 15:30

The Adept project is addressing the challenge of the energy-efficient use of parallel technologies. It builds on the expertise of software developers from high-performance computing to exploit parallelism for performance, and on the expertise of Embedded systems engineers in managing energy usage. Adept is developing a tool to guide software developers and help them model and predict the power consumption and performance of parallel software and hardware.

The strength of the HPC world lies primarily in software application parallelisation: concurrent computation is used to speed up the overall time that an application requires to run to completion. As a result, HPC software developers are also experts in parallel performance analysis and performance optimisation. The Embedded systems sector on the other hand excels in its management of energy usage because they are often constrained by fixed power and energy budgets. 

The strengths of one sector are the relative weaknesses of the other. In HPC, power management and power efficiency are in their infancy but becoming important, with HPC systems requiring more and more power. Embedded programmers are increasingly forced to use parallel computing techniques that are more familiar to HPC programmers because recent advances in low-power multi-core processors have widened the choice of hardware architectures for Embedded systems while energy and power budgets continue to be constrained.

Adept is investigating the implications of parallelism in programming models and algorithms, as well as choice of hardware, on energy and power consumption. 

It is important to gain a clear understanding of how factors such as redundant computations or algorithmic choices affect the power profile of a parallel application, or how this profile can be modified in a predictable way by off-loading compute-intensive parts of an application to low-power hardware. We are progressing the state of the art in application profiling, performance, and energy usage modelling in order to build a tool that integrates performance and energy consumption modelling for parallel Embedded and HPC systems.

Adept is advancing knowledge of how parallel software and hardware uses power. Being able to reduce the amount of power that is required to run large-scale applications on an HPC system will have a significant impact on the total cost of ownership and on the carbon-footprint of such a system. 

Adept also aims to increase programmer productivity by creating a tool that will be able to rapidly predict both the performance and the power usage of parallel systems, greatly reducing the need for speculative implementations to answer “what if?” questions during the software development process. This will enable developers to make informed decisions about hardware and software implementation that are economically viable in terms of performance and cost.

Adept benchmarks

Before implementing energy efficient algorithms or choosing the most power-performance efficient hardware, it is first necessary to understand and quantify any factors that dictate power consumption.

Benchmarks can be used to gain a deeper understanding of how implementation and architecture choices can impact on the overall efficiency of software. If we can identify the power usage profiles of computational patterns, it will become possible to optimise for energy and power in the same way we optimise for performance today. 

Adept has designed and implemented a set of benchmarks that can be used to measure both performance and power usage on a wide range of hardware architectures. The benchmarks are representative of the whole spectrum of parallel computing (from Embedded to HPC) and provide measurements that can be interpreted on a wide range of platforms. They also cover different levels of compute granularity, from single instructions and operations up to specific computational patterns in the form of kernels. 

More details can be found in our paper “Benchmarking for power consumption monitoring”, which was published as part of the International Conference on Energy-Aware High Performance Computing.

Find out more 

Project website: www.adept-project.eu

Twitter: @adept_project  

 

Images

Top: The low-tech approach to predicting energy usage!

Above: Visualising CPU and memory power measurements using the Adept Visualisation Tool.

ADEPT project partners: Uppsala University, Sweden; Ericsson AB, Sweden; Alpha Data Parallel Systems Ltd, UK; University of Ghent, Belgium. The project started in September 2013 and will run for 36 months.

Author

Michele Weiland, EPCC 

Comments

Hello Michele,

Could you please detail what do you mean by "off-loading compute-intensive parts of an application to low-power hardware".

And my second question would be: have you implemented a parallelizing pattern-aware software that reduces energy consumption, for certain parallel software patterns, compared to the serial execution?

Looking forward for your answer. Thank you in advance.

Kind regards,
Iulia

Hi Lulia,

By ‘off-loading compute-intensive parts of an application to low-power hardware’ we mean trying to identify portions of your code that could be efficiently run on, for instance, an FPGA or a low-power many-core architecture, and then porting that portion of the code to the new hardware. The idea is that you put your software, even just parts of it, on the hardware that will give it the best performance per Watt.

With regards to your second question: no, we have not looked at auto-parallelising pattern-aware software. We are experimenting with different parallel programming models and algorithms though. Take a look at the publications on our website for more details:
http://www.adept-project.eu/adept-articles.html