📣 Help Shape the Future of UKRI's Gateway to Research (GtR)

We're improving UKRI's Gateway to Research and are seeking your input! If you would be interested in being interviewed about the improvements we're making and to have your say about how we can make GtR more user-friendly, impactful, and effective for the Research and Innovation community, please email gateway@ukri.org.

Software Environment for Actionable & VVUQ-evaluated Exascale Applications (SEAVEA)

Lead Research Organisation: UNIVERSITY COLLEGE LONDON
Department Name: Chemistry

Abstract

Uncertainty quantification, verification and validation are crucial to establish the reliability and reproducibility of all forms of computer-based simulation. We propose to establish an open source and open development VVUQ toolkit optimised for efficient execution at current pre- and emerging exascale, which will raise new challenges and new opportunities for simulations in fields as diverse as fusion and climate modelling.

Computer simulation results are validated compared with experiment in several ways, ranging from qualitative to quantitative measures which apply a validation metric. Likewise, verification is concerned with confirmation that the mathematical model and corresponding algorithm have been coded correctly. Uncertainty quantification (UQ) is concerned with understanding the origins of and assessing the magnitudes of the errors which accompany computer simulations, whether epistemic or aleatoric.

VVUQ is necessary for any simulation that makes predictions in advance of an event to become actionable - that is, for its output to be useful in any form of decision-making process, from government interventions in pandemics to the choice of materials to combine for aircraft wing production. Here, exascale computing offers more opportunities to make actionable predictions.

Moreover, because VVUQ is intrinsically compute intensive due to its ensemble-based execution pattern, it too requires exascale resources, as well as advanced resource management strategies to efficiently manage the large numbers of concurrent runs necessary.

We propose to establish an open source and open development VVUQ toolkit optimised for efficient execution at current pre- and emerging exascale. This will include advanced approaches for surrogate modelling in order to minimise the expense and time needed to perform the most compute-intensive calculations and will demonstrate its efficiency gains for a diverse array of VVUQ workflows within multiple scientific applications, and on architecturally and geographically diverse emerging exascale environments.

The software developed, implemented and benchmarked in this project will become an open and invaluable asset to the UK ExCALIBUR community but also much more widely within UK and internationally as high-performance computing enters the exascale era.

The proposed exascale toolkit will be built on a combination of widely used tools and services which will be evolved to handle systems of increasing levels of complexity. These include components from the VECMA project (EasyVVUQ, FabSim3, QCG-PJ and EasySurrogate), as well as the UCL-Alan Turing Institute Multi-Output Gaussian Process Emulator (MOGP). We will apply these capabilities to several applications, including: (i) the UKAEA's tokamak fusion modelling use case for which a working software environment will be produced; (ii) weather and climate forecasting for the Met Office; (iii) turbulent flow simulation for environmental science; (iv) prediction of advanced materials properties of graphene-polymer based nanocomposites for aerospace applications; (v) high-fidelity patient-specific virtual human blood flow system for medical research; (vi) drug discovery; and (vii) human migration.

Publications

10 25 50

publication icon
Bieniek MK (2023) TIES 2.0: A Dual-Topology Open Source Relative Binding Free Energy Builder with Web Portal. in Journal of chemical information and modeling

publication icon
Coveney PV (2024) Artificial Intelligence Must Be Made More Scientific. in Journal of chemical information and modeling

publication icon
Edeling W (2023) On the Deep Active-Subspace Method in SIAM/ASA Journal on Uncertainty Quantification

publication icon
Ehara A (2023) AN ADAPTIVE STRATEGY FOR SEQUENTIAL DESIGNS OF MULTILEVEL COMPUTER EXPERIMENTS in International Journal for Uncertainty Quantification

 
Description We have found many Uncertainty Quantification studies require big supercomputers.

There is a new emphasis on 'VVUQ' studies for trusting predictions, making them actionable.
There are increased opportunities as high performance computing moves from petascale to exascale
Exploitation Route We support users currently outside of SEAVEA and organize dedicated workshops for them.

We work with other projects to integrate parts of SEAVEA Toolkit in their own software stacks.

We welcome external optimisation or VVUQ-related algorithms within the toolkit.

We are happy for other ExCALIBUR projects to provide advice on our development priorities.

We organize workshops, tutorials or hackathons with other ExCALIBUR projects.
Sectors Communities and Social Services/Policy

Digital/Communication/Information Technologies (including Software)

Education

Environment

Healthcare

Pharmaceuticals and Medical Biotechnology

URL https://www.seavea-project.org/
 
Title SEAVEA Toolkit 
Description The SEAVEA toolkit establishes a platform for verification, validation and uncertainty quantification (VVUQ), building on the work VECMAtk and UQ codes from the Alan Turing Institute (ATI). The goal is to provide tools that can be combined to capture complex scenarios, applied to applications in disparate domains, and used to run multiscale simulations on any desktop, cluster or supercomputing platform. 
Type Of Material Improvements to research infrastructure 
Year Produced 2022 
Provided To Others? Yes  
Impact The SEAVEA toolkit is open source and freely available to use with any application, using any programming language. 
URL https://www.seavea-project.org/seaveatk
 
Title FabSim3: An automation toolkit for verified simulations using high performance computing 
Description A common feature of computational modelling and simulation research is the need to perform many tasks in complex sequences to achieve a usable result. This will typically involve tasks such as preparing input data, pre-processing, running simulations on a local or remote machine, post-processing, and performing coupling communications, validations and/or optimisations. Tasks like these can involve manual steps which are time and effort intensive, especially when it involves the management of large ensemble runs. Additionally, human errors become more likely and numerous as the research work becomes more complex, increasing the risk of damaging the credibility of simulation results. Automation tools can help ensure the credibility of simulation results by reducing the manual time and effort required to perform these research tasks, by making more rigorous procedures tractable, and by reducing the probability of human error due to a reduced number of manual actions. In addition, efficiency gained through automation can help researchers to perform more research within the budget and effort constraints imposed by their projects. This paper presents the main software release of FabSim3, and explains how our automation toolkit can improve and simplify a range of tasks for researchers and application developers. FabSim3 helps to prepare, submit, execute, retrieve, and analyze simulation workflows. By providing a suitable level of abstraction, FabSim3 reduces the complexity of setting up and managing a large-scale simulation scenario, while still providing transparent access to the underlying layers for effective debugging. The tool also facilitates job submission and management (including staging and curation of files and environments) for a range of different supercomputing environments. Although FabSim3 itself is application-agnostic, it supports a provably extensible plugin system where users automate simulation and analysis workflows for their own application domains. To highlight this, we briefly describe a selection of these plugins and we demonstrate the efficiency of the toolkit in handling large ensemble workflows. 
Type Of Material Database/Collection of data 
Year Produced 2022 
Provided To Others? Yes  
URL https://data.mendeley.com/datasets/6nfrwy7ptj
 
Title FabSim3: An automation toolkit for verified simulations using high performance computing 
Description A common feature of computational modelling and simulation research is the need to perform many tasks in complex sequences to achieve a usable result. This will typically involve tasks such as preparing input data, pre-processing, running simulations on a local or remote machine, post-processing, and performing coupling communications, validations and/or optimisations. Tasks like these can involve manual steps which are time and effort intensive, especially when it involves the management of large ensemble runs. Additionally, human errors become more likely and numerous as the research work becomes more complex, increasing the risk of damaging the credibility of simulation results. Automation tools can help ensure the credibility of simulation results by reducing the manual time and effort required to perform these research tasks, by making more rigorous procedures tractable, and by reducing the probability of human error due to a reduced number of manual actions. In addition, efficiency gained through automation can help researchers to perform more research within the budget and effort constraints imposed by their projects. This paper presents the main software release of FabSim3, and explains how our automation toolkit can improve and simplify a range of tasks for researchers and application developers. FabSim3 helps to prepare, submit, execute, retrieve, and analyze simulation workflows. By providing a suitable level of abstraction, FabSim3 reduces the complexity of setting up and managing a large-scale simulation scenario, while still providing transparent access to the underlying layers for effective debugging. The tool also facilitates job submission and management (including staging and curation of files and environments) for a range of different supercomputing environments. Although FabSim3 itself is application-agnostic, it supports a provably extensible plugin system where users automate simulation and analysis workflows for their own application domains. To highlight this, we briefly describe a selection of these plugins and we demonstrate the efficiency of the toolkit in handling large ensemble workflows. 
Type Of Material Database/Collection of data 
Year Produced 2022 
Provided To Others? Yes  
URL https://data.mendeley.com/datasets/6nfrwy7ptj/1
 
Description RADICAL-Cybertools 
Organisation Brookhaven National Laboratory
Country United States 
Sector Public 
PI Contribution We teamed up with the RADICAL-Cybertools development team based in the Brookhaven National Labs, USA to collaborate with us for the SEAVEA project to deal with the technical challenges found in high performance computing to schedule and execute large number of jobs inside the system resource allocation
Collaborator Contribution provided us for regular software releases and access for the RADICAL-Cybertools that currently consists of three components: RADICAL-SAGA: a standards-based interface that provides basic interoperability across a range of computing middleware; RADICAL-Pilot: a scalable and flexible Pilot-Job system that provides flexible application-level resource management capabilities, and Ensemble Toolkit that simplifies the ability to implement ensemble-based applications.
Impact The released two versions of the RADICAL-Cybertools package for us to access and use in our research. We run monthly meetings with them to get regular updates for progress of work.
Start Year 2021
 
Description SEAVEA: UQ in Fluid Turbulence 
Organisation Imperial College London
Country United Kingdom 
Sector Academic/University 
PI Contribution We developed the SEAVEA Toolkit to this collaboration for the Imperial team to use to quantifying uncertainties in direct numerical simulations of a turbulent channel flow by a post-doc research fellow.
Collaborator Contribution To facilitate the non-intrusive forward UQ analysis, the open-source EasyVVUQ package developed by UCL was used by my partner Imperial College to provide integrated capability for sampling, pre-processing, execution, post-processing, and analysis of the computational campaign.
Impact A paper published with the title of "Quantifying uncertainties in direct numerical simulations of a turbulent channel flow".
Start Year 2021