EVERSE: driving improvements in research software quality

This is an account of what the project is setting of to achieve, of the progress so far and of  the directions its work packages are taking. 

A European network of Research Software Quality

A core goal of the EVERSE consortium is to create a sustainable network that will outlive the project itself and continue driving improvements in research software quality across Europe and internationally. To achieve this, the project is focused on establishing the European network of Research Software Quality. The team has launched a project website, newsletter, and social media channels to engage the broader community.

Practical plans are underway to expand and strengthen this network. The project is exploring collaborations with organisations like the Research Software Alliance (ReSA) and the Science for Africa (SFA) Foundation. These collaborations aim to expand the EVERSE network globally, and learn about how other extra-European communities are addressing research software quality.

EVERSE plans to participate in and organise satellite events at international conferences to further its goals. The project will also host online webinars on software best practices for various communities, for example data stewards. These activities aim to promote knowledge exchange, showcase EVERSE tools and frameworks, and gather input from diverse research communities.

A community-driven knowledge-hub

A key component of EVERSE is the Research Software Quality toolkit (RSQkit), a knowledge base for promoting high-quality software and code across disciplines. Inspired by initiatives like the ELIXIR Research Data Management Kit, RSQkit serves as a gateway to catalogues, services, standards, and best practices for research software quality. It is tailored to the needs of various users, including software developers, researchers, infrastructure providers, and policymakers, ensuring it addresses diverse needs throughout the research software lifecycle.

Defining software excellence

EVERSE is developing a reference model for research software excellence. This model considers multiple dimensions including technical quality, FAIR principles, openness, and sustainability. It examines software excellence through various perspectives, such as software complexity, its role in the research lifecycle, and domain-specific needs. The goal is to create a comprehensive framework to guide researchers in improving their software across all fields and scales.

Key categories for software quality have been identified, including source code metrics, performance metrics, testing practices, and FAIR principles. These are being mapped to practical guidelines and tools to support researchers in enhancing their software quality.

Gathering best practices

Work Package 2 (WP2) is conducting a landscape analysis of existing best practices for research software development across scientific domains. The initial phase involves a survey circulated to a restricted group of users, which will be followed by a refined survey distributed more widely to scientific communities. 

WP2 has noted that many excellent practices already exist in different communities. Some of the emerging best practices being identified include modularisation of code, implementation of comprehensive testing strategies, use of version control systems, clear documentation practices, adherence to FAIR principles for research software, and implementation of code review processes. 

Its task is to consolidate these practices, identify core principles that apply across domains, and make them accessible to all researchers. This approach aims to create a comprehensive, cross-domain set of best practices for research software development, building upon existing knowledge while ensuring broad applicability and accessibility.

Tools and services

Work Package 3 (WP3) is focused on identifying and integrating tools and services to support software quality assessment and improvement. It is creating a catalogue of existing tools and exploring ways to combine them into common frameworks that can be easily adopted by different research communities. Some specific tools and services being catalogued include FAIR assessment tools, software metadata enrichment services, and CI/CD (Continuous Integration/Continuous Deployment) pipelines. 

The approach is not to reinvent the wheel, but to make it easier for researchers to find and use the right tools for their needs, whether that's improving code quality, ensuring FAIR principles, or measuring impact. This activity of EVERSE has synergies and links with FAIR Impact, another ongoing project at EPCC.

Science clusters and pilot projects

Work Package 4 (WP4) is setting up pilot projects across the five EOSC Science Clusters to test and refine the EVERSE framework for research software quality. Pilots have been presented from the ESCAPE cluster, including high-energy physics software like xAODAnaHelpers, BAILER-IT, and ACTS, as well as from the PANOSC cluster, including the PiConGPU simulation tool and Alpaka hardware abstraction library. 

These pilots represent different tiers of research software, from analysis code to research infrastructure. WP4 is developing a process for software quality assessment that will incorporate metrics from WP2 and tools from WP3, with the real-world examples from the pilots being crucial for validating the EVERSE framework and tools across different scientific domains.

Capacity building and recognition

Work Package 5 (WP5) focuses on capacity building and recognition for research software. The team is working on three main tasks: 

• Collecting and curating training resources aligned with best practices for research software quality
• Developing a framework for recognising trainers and Research Software Engineers (RSEs) 
• Establishing long-term training activities. 

WP5 is compiling a comprehensive list of existing training materials and recognition mechanisms, which will inform the development of the Research Software Quality toolkit (RSQkit). The team is also exploring ways to integrate these resources with existing platforms like TeSS and is considering the development of a curriculum for software engineers. 

The work package aims to address the full spectrum of the research software lifecycle and is collaborating closely with other work packages to ensure alignment with the overall EVERSE goals.

Looking ahead

As EVERSE moves into its next phase, the focus will be on further developing the RSQkit, refining the software excellence framework, and expanding the pilot activities. Several key milestones are on the horizon, including the completion of the initial best practices landscaping results (Milestone M2.1) and the first evaluation workshop of tools and services against initial best practices (Milestone MS3.1). 

The project is also gearing up for the EOSC Symposium later this month, where it plans to showcase the RSQkit. Additionally, the team is considering organising contentathons or writing sprints to populate the RSQkit with best practices, though these are still in the planning stages. The project is also planning increased engagement with the broader research community through events, training activities, and collaboration with other EOSC initiatives.

The EVERSE consortium

The consortium is led by the Ethniko Kentro Erevnas Kai Technologikis Anaptyxis (CERTH) in Thessaloniki, Greece, and includes universities and research institutions from across Europe. EPCC is a partner, with the Software Sustainability Institute as the involved group. The EVERSE consortium encompasses five EOSC Science Clusters: ENVRI-FAIR (earth and environmental sciences), ESCAPE (high energy particle physics and astronomy), PANOSC (photon and neutron science), SSHOC (social sciences & humanities), and EOSC-Life (life sciences). 

The EVERSE project will run from March 2024 to February 2027.  

EVERSE website: https://everse.software

References

[1] Wilkinson, M., Dumontier, M., Aalbersberg, I. et al. The FAIR Guiding Principles for scientific data management and stewardship. Sci Data 3, 160018 (2016). https://doi.org/10.1038/sdata.2016.18

[2] Chue Hong, N., Katz, D.S., Barker, M. et al. FAIR Principles for Research Software version 1.0. (FAIR4RS Principles v1.0). Research Data Alliance (2022). https://doi.org/10.15497/RDA00068

[3] Lamprecht, A., Garcia, L., Kuzak, M. et al. Towards FAIR principles for research software. Data Science 3, 37-59 (2020). https://doi.org/10.3233/DS-190026

[4] Barker, M., Chue Hong, N.P., Katz, D.S. et al. Introducing the FAIR Principles for research software. Sci Data 9, 622 (2022). https://doi.org/10.1038/s41597-022-01710-x

[5] Honeyman, T., Katz, D.S., Lammey, R. et al. A pathway towards multidimensional research software quality. arXiv:2303.06611 [cs.SE] (2023). https://doi.org/10.5281/zenodo.4940273