CamFort: Automated evolution and verification of computational science models

Scientific models play a vital role in science and policy making. Many models are now expressed as complex computer programs which are often the result of decades of research and development, possibly involving multiple researchers or teams. This has lead to significant investment in maintaining these models and evolving them to use modern programming approaches or to work efficiently on new hardware platforms (such as cloud computing resources). However, the complexity of these models makes maintenance and evolution difficult. In particular, changing a complex model's code whilst ensuring it produces the same results is difficult; maintenance/evolution of complex models is often error prone.

The complexity of a piece of software can be classified as either intrinsic or accidental. Intrinsic complexity is an essential reflection of the complexity inherent in the problem and solution at hand. Alternatively, accidental complexity arises from the particular programming language, design or tools used to implement the solution. Many of the research contributions of programming language design and software engineering have been aimed at reducing the accidental complexity of software. However, many of these approaches have not been targetted at scientific computing.

There is now a need to develop these contributions so that they meet the needs of scientists. Addressing these needs will provide huge benefits to science and policy through increased productivity and trust in models. Our collaborations with leading research groups in science have highlighted the huge existing investments in established models. We are therefore aiming to support the evolution, rather than replacement, of existing code and working practices. Our goal is to apply cutting edge programming language and software engineering research to help develop "sustainable" software, which maintains its value over generations of researcher. Our focus is on models developed in the Fortran language, as this remains a dominant programming language in scientific computing, owing in part to its longevity.

We will provide practical tools which scientists can use to reduce the accidental complexity of models through evolving a code base, as well as tools for automatically verifying that any maintenance/evolution activities preserve the models behaviour. We will develop new mechanisms for program comprehension and transformation in order to bring effective techniques from programming language design and software engineering across the chasm to scientific computing. Ultimately, reducing the effort to maintain and evolve code will free-up scientists to focus on the core aspects of the science, and will lead to models that are more easily communicated, disseminated, and reused between researchers, supporting core ideals of science.

Computer modelling is now a large part of science and engineering research. However, there are many challenges, e.g., verification, parallelisation, and maintenance. Despite significant programming language and software engineering research to address these concerns, very few results are applied to science; there is little interaction between natural science and programming research communities. Given the importance, the benefits, and the challenges of computational science, it is critical that foundational programming language and software engineering research be targeted at computational science. There is a great opportunity for progress and impact if these communities interact. Our project does exactly this, addressing problems of verified model evolution. However, many more interdisciplinary projects are needed. Part of our project's impact is therefore in promoting a cultural shift to computer science engagement with the sciences, which is critical to the health of future scientific research.

In programming language and software engineering research, our project will provide contributions to various areas such as (1) parsing - accommodating languages with embedded fragments of an unrecognised language, e.g. pre-processor code; (2) program analysis - for advanced refactorings and new verification techniques, e.g. advanced resource analysis and high-level programming pattern detection; (3) verification - techniques for differential testing of program fragments (e.g., before and after refactoring). More broadly, the interdisciplinary nature of this project will provide knowledge and data on the programming needs of computational scientists, which will aid future language research into new programming approaches for science.

As noted by the 'Programming languages and compilers' research area of the EPSRC ICT theme, a lack of implementation effort in this area has held-back impact. Our project includes significant implementation and engagement effort to ensure adoption, impact, and appropriate evaluation of the new programming tools.

Our work benefits scientists and engineers in academia and industry who are involved in computer modelling using Fortran. Our tools will aid the evolution of existing code base which may use custom pre-processors. Furthermore, our tools for automated testing will improve verification of evolution activities, reducing bugs and the effort required to find and correct them. In the short term, our tools will have immediate impact on our collaborators as our research is evaluated in the context of active scientific research. Our collaborators include the Cambridge Centre for Climate Change Mitigation Research (4CMR), the Met Office (the Unified Model), the Centre for Climate Science (Department of Geography), and the Department of Chemistry. The scope of our impact is broadened by dissemination activities (workshops, seminars) and providing the tools free online (and open source).

On medium to long-term scales, our work will indirectly benefit other aspects of computer modelling by reducing maintenance and verification efforts and thus allowing scientists to focus on other aspects, e.g. underlying science and validation of models.

Scientific research benefits society mainly through technology (when science is applied via engineering) and policy (informed by predictions). Since our research aims to improve computer modelling in science and engineering, we expect more long-term, indirect impact, although this is difficult to quantify given the technical nature of our research. Nonetheless, many of our collaborators are climate scientists and thus there is route to social, environmental, and policy-based impact. For example, the 4CMR has a number of ongoing projects with policy makers (including OFWAT, the Environment Agency, Department of Energy and Climate Change, Highways Agency, and British Land); our collaboration with 4CMR provides an opportunity for indirect policy/social impact.