Agony and ExTASY
Posted: 8 Jun 2015 | 16:41
Over 18 months ago, we kicked off the ExTASY project. Our lofty aim was to tackle one of the 'Grand Challenges' of biochemistry - effective sampling of complex biomolecules using Molecular Dynamics to provide a physical understanding of their function and behaviour.
At a project review meeting last September I confidently planned for a few weeks' worth of testing, which would lead to the first release of our software. Some 9 months and over a hundred issue reports later, we have finally made a public release of our code! Perhaps not as epic a mis-prediction a TJ Watson's "world market for five computers", but still a major underestimation of the challenges we have now overcome. So why did it prove to be so hard?
Firstly, here's what we have achieved in ExTASY version 0.1:
- A Python tool that can be installed on Linux or Mac OS X, which features two MD sampling workflows, coupling MD codes (GROMACS & AMBER) and MD analysis tools (COCO and LSDMap)
- Transparent execution on remote HPC services (ARCHER and Stampede)
- Extensive documentation including a user guide and tutorial
- Beta testing with several potential user groups (internal and external to the project).
What took so long?
The primary obstacle we ran into time and again was that software that integrates several (distributed) components has to be able to cope with not only error scenarios that occur due to each tool, but also with the interactions between them. Throw in two very different computing platforms to which we provide an abstract interface (but are exposed to their unique foibles) plus client deployment on Linux and Mac OS, and the possibilities for failure grows combinatorially. In the end we had 63 different test scenarios which we were required to pass before the release could be made, each of which was manually tested by one of our distributed team members!
One lesson from this is to automate as much testing as possible, as early as possible - in the end we made use of Jenkins for automated testing, which reduced the turnaround time for testing new builds from days to minutes.
The second reason for the release date being pushed further and further back was that the exact functionality contained in the release was continually modified as we went along. It's hard to make it over the line when the line keeps moving further away! At several points there were misunderstandings between the end-user scientists (who of course hadn't precisely enough defined their requirements!) and the software team (who made incorrect assumptions about what the users need!) that resulted in time-wasting re-implementation (and re-design).
At some level this is expected in the development of research software - part of the research is discovering what we need to build. However, we have learned to be much clearer from the outset about the requirements of future releases, which should shorten the development & test cycle.
After finding and fixing even more issues during Beta testing (mostly resulting in improved documentation), we were able to make the 0.1 release of ExTASY in May. Despite (or perhaps because of) the difficulties in development, it's a great feeling to finally see the end result!
We are only a little over halfway through the ExTASY project now, and the second half of 2015 looks like it will be busy. In the pipeline is the next release of the ExTASY tool, which will move away from providing a static workflow tool to a flexible Python API to allow end-users to define their own workflows. We will also make the first release of the Molecular Integration Simulation Toolkit (MIST), and run a CECAM workshop where we will introduce our tools to the wider community!
I hope to write another blog post after our next release - in a few months time...
Image: Free energy landscape of alanine-12, computed using DM-d-MD (Diffusion Map-directed-Molecular Dynamics) showing selected stable conformations, from J. Preto & C. Clementi, PCCP, 2014, 16