Software and Skills for Large-Scale Computing: collecting evidence to develop a National Research Software Strategy

Software has revolutionised the way that we conduct research, pervading all aspects of the research lifecycle. From machine-learning and big data to modelling and simulation, from management of experiments to record keeping and reproducibility, research is increasingly powered by software and digital technologies have become vital tools and techniques for the modern researcher. These tools and techniques require new skills and, often, new forms of research collaboration.

Recently, two reports laid the foundations for understanding the requirements for software and skills to support modern research. The Government Office for Science published a report titled "Large-scale computing: the case for greater UK coordination" which reviewed the UK's large-scale computing ecosystem and the interdependency of hardware, software and skills. This report highlighted the "strong case for continued public investment in large-scale computing" while noting that "investment in computer hardware alone will not be sufficient". It made strong recommendations on investment in software and skills, noting that "high-quality software is fundamental to realising the benefits of investments in computing. Software must be fit for purpose and be regularly tested, updated, and archived" and "there is an acute shortage of large-scale computing professionals, including system architects, data engineers, system operations professionals and software engineers. Exploitation of advanced computing requires skilled cross-disciplinary teams". Separately, the "ExCALIBUR Research Software Engineer Knowledge Integration Landscape Review" provided a comprehensive review of the skills required by Research Software Engineers (RSEs) in High Performance Computing (HPC) and their future training needs.

Building on these reports and the work of the Software Sustainability Institute, the goal of this project is to deliver a better understanding of the software and skills required for large-scale research computing (including HPC, HTC/cloud, AI/ML and data science) and recommendations for how policies and support for these should be structured.

This work will involve a two stage, mixed methods approach. The first stage, encompassing the first four project months, will comprise data gathering through a series of approaches including interviews, focus groups, workshops and surveys, as well as a literature review. The second stage, encompassing the last three project months, will comprise data analysis and the distillation of the data into a report on (and recommendations for) the support required for software and skills to support large-scale research computing. This study will contribute to the evidence base that can be used to develop a National Research Software Strategy for the UK and directly addresses Objective 2 of EPSRC's phase one DRI Objectives: Building the UK's skills base and software development community required for the continued operation of high quality HPC services.