Software Environment for Actionable & VVUQ-evaluated Exascale Applications (SEAVEA)
Lead Research Organisation:
UNIVERSITY COLLEGE LONDON
Department Name: Chemistry
Abstract
Uncertainty quantification, verification and validation are crucial to establish the reliability and reproducibility of all forms of computer-based simulation. We propose to establish an open source and open development VVUQ toolkit optimised for efficient execution at current pre- and emerging exascale, which will raise new challenges and new opportunities for simulations in fields as diverse as fusion and climate modelling.
Computer simulation results are validated compared with experiment in several ways, ranging from qualitative to quantitative measures which apply a validation metric. Likewise, verification is concerned with confirmation that the mathematical model and corresponding algorithm have been coded correctly. Uncertainty quantification (UQ) is concerned with understanding the origins of and assessing the magnitudes of the errors which accompany computer simulations, whether epistemic or aleatoric.
VVUQ is necessary for any simulation that makes predictions in advance of an event to become actionable - that is, for its output to be useful in any form of decision-making process, from government interventions in pandemics to the choice of materials to combine for aircraft wing production. Here, exascale computing offers more opportunities to make actionable predictions.
Moreover, because VVUQ is intrinsically compute intensive due to its ensemble-based execution pattern, it too requires exascale resources, as well as advanced resource management strategies to efficiently manage the large numbers of concurrent runs necessary.
We propose to establish an open source and open development VVUQ toolkit optimised for efficient execution at current pre- and emerging exascale. This will include advanced approaches for surrogate modelling in order to minimise the expense and time needed to perform the most compute-intensive calculations and will demonstrate its efficiency gains for a diverse array of VVUQ workflows within multiple scientific applications, and on architecturally and geographically diverse emerging exascale environments.
The software developed, implemented and benchmarked in this project will become an open and invaluable asset to the UK ExCALIBUR community but also much more widely within UK and internationally as high-performance computing enters the exascale era.
The proposed exascale toolkit will be built on a combination of widely used tools and services which will be evolved to handle systems of increasing levels of complexity. These include components from the VECMA project (EasyVVUQ, FabSim3, QCG-PJ and EasySurrogate), as well as the UCL-Alan Turing Institute Multi-Output Gaussian Process Emulator (MOGP). We will apply these capabilities to several applications, including: (i) the UKAEA's tokamak fusion modelling use case for which a working software environment will be produced; (ii) weather and climate forecasting for the Met Office; (iii) turbulent flow simulation for environmental science; (iv) prediction of advanced materials properties of graphene-polymer based nanocomposites for aerospace applications; (v) high-fidelity patient-specific virtual human blood flow system for medical research; (vi) drug discovery; and (vii) human migration.
Computer simulation results are validated compared with experiment in several ways, ranging from qualitative to quantitative measures which apply a validation metric. Likewise, verification is concerned with confirmation that the mathematical model and corresponding algorithm have been coded correctly. Uncertainty quantification (UQ) is concerned with understanding the origins of and assessing the magnitudes of the errors which accompany computer simulations, whether epistemic or aleatoric.
VVUQ is necessary for any simulation that makes predictions in advance of an event to become actionable - that is, for its output to be useful in any form of decision-making process, from government interventions in pandemics to the choice of materials to combine for aircraft wing production. Here, exascale computing offers more opportunities to make actionable predictions.
Moreover, because VVUQ is intrinsically compute intensive due to its ensemble-based execution pattern, it too requires exascale resources, as well as advanced resource management strategies to efficiently manage the large numbers of concurrent runs necessary.
We propose to establish an open source and open development VVUQ toolkit optimised for efficient execution at current pre- and emerging exascale. This will include advanced approaches for surrogate modelling in order to minimise the expense and time needed to perform the most compute-intensive calculations and will demonstrate its efficiency gains for a diverse array of VVUQ workflows within multiple scientific applications, and on architecturally and geographically diverse emerging exascale environments.
The software developed, implemented and benchmarked in this project will become an open and invaluable asset to the UK ExCALIBUR community but also much more widely within UK and internationally as high-performance computing enters the exascale era.
The proposed exascale toolkit will be built on a combination of widely used tools and services which will be evolved to handle systems of increasing levels of complexity. These include components from the VECMA project (EasyVVUQ, FabSim3, QCG-PJ and EasySurrogate), as well as the UCL-Alan Turing Institute Multi-Output Gaussian Process Emulator (MOGP). We will apply these capabilities to several applications, including: (i) the UKAEA's tokamak fusion modelling use case for which a working software environment will be produced; (ii) weather and climate forecasting for the Met Office; (iii) turbulent flow simulation for environmental science; (iv) prediction of advanced materials properties of graphene-polymer based nanocomposites for aerospace applications; (v) high-fidelity patient-specific virtual human blood flow system for medical research; (vi) drug discovery; and (vii) human migration.
Organisations
- UNIVERSITY COLLEGE LONDON (Lead Research Organisation)
- IMPERIAL COLLEGE LONDON (Collaboration)
- Brookhaven National Laboratory (Collaboration)
- Save the Children (Project Partner)
- Polish Academy of Sciences (Project Partner)
- Imperial College London (Project Partner)
- RIKEN (Project Partner)
- Centrum Wiskunde & Informatica (Project Partner)
- Argonne National Laboratory (Project Partner)
- University of Cambridge (Project Partner)
- Max Planck Institutes (Project Partner)
- CCFE/UKAEA (Project Partner)
- Rutgers, The State University of New Jersey (Project Partner)
Publications
Ahmad K
(2023)
Structure and dynamics of an archetypal DNA nanoarchitecture revealed via cryo-EM and molecular dynamics simulations.
in Nature communications
Bhati A
(2022)
Large Scale Study of Ligand-Protein Relative Binding Free Energy Calculations: Actionable Predictions from Statistically Robust Protocols
in Journal of Chemical Theory and Computation
Bhati AP
(2023)
Long Time Scale Ensemble Methods in Molecular Dynamics: Ligand-Protein Interactions and Allostery in SARS-CoV-2 Targets.
in Journal of chemical theory and computation
Bieniek MK
(2023)
TIES 2.0: A Dual-Topology Open Source Relative Binding Free Energy Builder with Web Portal.
in Journal of chemical information and modeling
Coveney P
(2024)
Sharkovskii's theorem and the limits of digital computers for the simulation of chaotic dynamical systems
in Journal of Computational Science
Coveney PV
(2024)
Artificial Intelligence Must Be Made More Scientific.
in Journal of chemical information and modeling
Edeling W
(2024)
Global ranking of the sensitivity of interaction potential contributions within classical molecular dynamics force fields
in npj Computational Materials
Edeling W
(2023)
On the Deep Active-Subspace Method
in SIAM/ASA Journal on Uncertainty Quantification
Ehara A
(2023)
AN ADAPTIVE STRATEGY FOR SEQUENTIAL DESIGNS OF MULTILEVEL COMPUTER EXPERIMENTS
in International Journal for Uncertainty Quantification
Ehara A
(2022)
Multi-level emulation of tsunami simulations over Cilacap, South Java, Indonesia
in Computational Geosciences
| Description | We have found many Uncertainty Quantification studies require big supercomputers. There is a new emphasis on 'VVUQ' studies for trusting predictions, making them actionable. There are increased opportunities as high performance computing moves from petascale to exascale |
| Exploitation Route | We support users currently outside of SEAVEA and organize dedicated workshops for them. We work with other projects to integrate parts of SEAVEA Toolkit in their own software stacks. We welcome external optimisation or VVUQ-related algorithms within the toolkit. We are happy for other ExCALIBUR projects to provide advice on our development priorities. We organize workshops, tutorials or hackathons with other ExCALIBUR projects. |
| Sectors | Communities and Social Services/Policy Digital/Communication/Information Technologies (including Software) Education Environment Healthcare Pharmaceuticals and Medical Biotechnology |
| URL | https://www.seavea-project.org/ |
| Title | SEAVEA Toolkit |
| Description | The SEAVEA toolkit establishes a platform for verification, validation and uncertainty quantification (VVUQ), building on the work VECMAtk and UQ codes from the Alan Turing Institute (ATI). The goal is to provide tools that can be combined to capture complex scenarios, applied to applications in disparate domains, and used to run multiscale simulations on any desktop, cluster or supercomputing platform. |
| Type Of Material | Improvements to research infrastructure |
| Year Produced | 2022 |
| Provided To Others? | Yes |
| Impact | The SEAVEA toolkit is open source and freely available to use with any application, using any programming language. |
| URL | https://www.seavea-project.org/seaveatk |
| Title | FabSim3: An automation toolkit for verified simulations using high performance computing |
| Description | A common feature of computational modelling and simulation research is the need to perform many tasks in complex sequences to achieve a usable result. This will typically involve tasks such as preparing input data, pre-processing, running simulations on a local or remote machine, post-processing, and performing coupling communications, validations and/or optimisations. Tasks like these can involve manual steps which are time and effort intensive, especially when it involves the management of large ensemble runs. Additionally, human errors become more likely and numerous as the research work becomes more complex, increasing the risk of damaging the credibility of simulation results. Automation tools can help ensure the credibility of simulation results by reducing the manual time and effort required to perform these research tasks, by making more rigorous procedures tractable, and by reducing the probability of human error due to a reduced number of manual actions. In addition, efficiency gained through automation can help researchers to perform more research within the budget and effort constraints imposed by their projects. This paper presents the main software release of FabSim3, and explains how our automation toolkit can improve and simplify a range of tasks for researchers and application developers. FabSim3 helps to prepare, submit, execute, retrieve, and analyze simulation workflows. By providing a suitable level of abstraction, FabSim3 reduces the complexity of setting up and managing a large-scale simulation scenario, while still providing transparent access to the underlying layers for effective debugging. The tool also facilitates job submission and management (including staging and curation of files and environments) for a range of different supercomputing environments. Although FabSim3 itself is application-agnostic, it supports a provably extensible plugin system where users automate simulation and analysis workflows for their own application domains. To highlight this, we briefly describe a selection of these plugins and we demonstrate the efficiency of the toolkit in handling large ensemble workflows. |
| Type Of Material | Database/Collection of data |
| Year Produced | 2022 |
| Provided To Others? | Yes |
| URL | https://data.mendeley.com/datasets/6nfrwy7ptj |
| Title | FabSim3: An automation toolkit for verified simulations using high performance computing |
| Description | A common feature of computational modelling and simulation research is the need to perform many tasks in complex sequences to achieve a usable result. This will typically involve tasks such as preparing input data, pre-processing, running simulations on a local or remote machine, post-processing, and performing coupling communications, validations and/or optimisations. Tasks like these can involve manual steps which are time and effort intensive, especially when it involves the management of large ensemble runs. Additionally, human errors become more likely and numerous as the research work becomes more complex, increasing the risk of damaging the credibility of simulation results. Automation tools can help ensure the credibility of simulation results by reducing the manual time and effort required to perform these research tasks, by making more rigorous procedures tractable, and by reducing the probability of human error due to a reduced number of manual actions. In addition, efficiency gained through automation can help researchers to perform more research within the budget and effort constraints imposed by their projects. This paper presents the main software release of FabSim3, and explains how our automation toolkit can improve and simplify a range of tasks for researchers and application developers. FabSim3 helps to prepare, submit, execute, retrieve, and analyze simulation workflows. By providing a suitable level of abstraction, FabSim3 reduces the complexity of setting up and managing a large-scale simulation scenario, while still providing transparent access to the underlying layers for effective debugging. The tool also facilitates job submission and management (including staging and curation of files and environments) for a range of different supercomputing environments. Although FabSim3 itself is application-agnostic, it supports a provably extensible plugin system where users automate simulation and analysis workflows for their own application domains. To highlight this, we briefly describe a selection of these plugins and we demonstrate the efficiency of the toolkit in handling large ensemble workflows. |
| Type Of Material | Database/Collection of data |
| Year Produced | 2022 |
| Provided To Others? | Yes |
| URL | https://data.mendeley.com/datasets/6nfrwy7ptj/1 |
| Description | RADICAL-Cybertools |
| Organisation | Brookhaven National Laboratory |
| Country | United States |
| Sector | Public |
| PI Contribution | We teamed up with the RADICAL-Cybertools development team based in the Brookhaven National Labs, USA to collaborate with us for the SEAVEA project to deal with the technical challenges found in high performance computing to schedule and execute large number of jobs inside the system resource allocation |
| Collaborator Contribution | provided us for regular software releases and access for the RADICAL-Cybertools that currently consists of three components: RADICAL-SAGA: a standards-based interface that provides basic interoperability across a range of computing middleware; RADICAL-Pilot: a scalable and flexible Pilot-Job system that provides flexible application-level resource management capabilities, and Ensemble Toolkit that simplifies the ability to implement ensemble-based applications. |
| Impact | The released two versions of the RADICAL-Cybertools package for us to access and use in our research. We run monthly meetings with them to get regular updates for progress of work. |
| Start Year | 2021 |
| Description | SEAVEA: UQ in Fluid Turbulence |
| Organisation | Imperial College London |
| Country | United Kingdom |
| Sector | Academic/University |
| PI Contribution | We developed the SEAVEA Toolkit to this collaboration for the Imperial team to use to quantifying uncertainties in direct numerical simulations of a turbulent channel flow by a post-doc research fellow. |
| Collaborator Contribution | To facilitate the non-intrusive forward UQ analysis, the open-source EasyVVUQ package developed by UCL was used by my partner Imperial College to provide integrated capability for sampling, pre-processing, execution, post-processing, and analysis of the computational campaign. |
| Impact | A paper published with the title of "Quantifying uncertainties in direct numerical simulations of a turbulent channel flow". |
| Start Year | 2021 |
