Coding Smart in Academia: Evidence-Based Software Engineering Approaches for High-Quality Research Software Projects

Research software describes any code written or software used in the process of generating results for a publication - for example, a script used to process raw data or carry out a statistical analysis - and is commonly written by researchers without a software engineering background or extensive experience.

Inaccuracies or inefficiencies in research software impact the research that relies on it. While other tools used for research might be calibrated, tested, or evaluated in detail, software can often be overlooked, and problems might not be identified.


This project will investigate research software from three key areas of focus (software engineering techniques, software quality, and research software project properties), to gain a better understanding of the software engineering techniques used to develop it, how to best assess the quality of research software and which research software project factors are influential in developing high quality outputs. It will consider whether insights or techniques used widely within traditional software development could be of use within research software development.

The project will also gather empirical data from selected research software projects to identify evidence-based recommendations for the research software engineering community to help build better-quality research software.

To achieve these aims, this project will assess methods for evaluating research software quality and consider the contexts and properties of projects developing research software, identifying how these might impact the software generated.

Most previous research and literature have dealt with 'scientific software'. This project will address the wider area of 'research software' - a more inclusive and interdisciplinary approach which also captures practices from software used for research within the arts and humanities.

Previous in-depth studies took place before widespread institutional support of research software projects and the emergence of a 'research software engineer' role. This study will update the literature on whether these landscape shifts indicate changes in practice.

Many existing studies covering research software offer recommendations regarding best practices, however, little empirical evidence (particularly qualitative experimental data) is given to demonstrate that these recommendations are effective, while the range of research software projects they can be validly applied to is not clearly shown.

To address this evidence gap, this project will identify potential improvements and more effective software engineering methods or techniques currently under-used within research software projects.

These suggested interventions will be tested for their impacts within a representative set of research software projects. A 'control' group of closely matched projects (for example: similar software engineering technique usage, baseline research software quality, and project contexts such as development team sizes) will not receive an intervention. Data about software development practices and project contexts will be gathered from both groups of participating projects via surveys and interviews over the study period. Research software quality within these selected projects will also be measured throughout the experimental period to assess the impacts of these interventions on software quality. The recommended interventions will be refined based on this experimental testing, generating robust evidence-based and verified solutions to current research software challenges encountered by a range of research software projects. The outcomes of this study will allow researchers to identify which software engineering techniques and methods are most effective and would suit their project (listing known benefits or drawbacks) and demonstrate how to assess and improve the quality of their research software, with all suggestions strongly supported by data.