Exascale Computing for System-Level Engineering: Design, Optimisation and Resilience
Lead Research Organisation:
University College London
Department Name: Mechanical Engineering
Abstract
The arrival of exascale computers will open new frontiers in our ability to simulate highly complex engineered and natural systems. This will create new opportunities for the design and optimisation of new, highly integrated engineered systems for the future. It will also allow the development of 'digital twins' of complex natural systems, such has the human body and coastal/river regions, that will allow is to explore and manage engineering-led interventions in personalised healthcare and management of the natural environment.
The exascale computers of the future will be highly parallel with hundreds of thousands, or millions of processes, working collectively. Exploiting this remarkable level of parallelism will require dramatic advances in the mathematics, numerical methods, software engineering and software tools that underpin simulation, and will depend on experts in each of these areas coming together. The simulation of the different but tightly coupled physical processes that characterises complex engineered and natural systems poses additional challenges of coordinating the simulation of multiple processes, such as the noise created by an airflow flow around a moving structure under the influence of a magnetic field, or the fluid, solid, electrical and chemical interactions a human body.
This project brings together of working group of experts from computer science, mathematics and engineering to address the challenge of how to simulate coupled physical process at a system level on future exascale systems. It will also address how to integrate into the simulation process the vast quantity of data that can be collected from real systems, how to assess uncertainties and how to interpret the vast quantities of data that exascale simulations will generate. The working group will formulate roadmaps for enabling research for exascale computing, and support research software engineer training for exascale-ready software skills.
The exascale computers of the future will be highly parallel with hundreds of thousands, or millions of processes, working collectively. Exploiting this remarkable level of parallelism will require dramatic advances in the mathematics, numerical methods, software engineering and software tools that underpin simulation, and will depend on experts in each of these areas coming together. The simulation of the different but tightly coupled physical processes that characterises complex engineered and natural systems poses additional challenges of coordinating the simulation of multiple processes, such as the noise created by an airflow flow around a moving structure under the influence of a magnetic field, or the fluid, solid, electrical and chemical interactions a human body.
This project brings together of working group of experts from computer science, mathematics and engineering to address the challenge of how to simulate coupled physical process at a system level on future exascale systems. It will also address how to integrate into the simulation process the vast quantity of data that can be collected from real systems, how to assess uncertainties and how to interpret the vast quantities of data that exascale simulations will generate. The working group will formulate roadmaps for enabling research for exascale computing, and support research software engineer training for exascale-ready software skills.
Publications
Behrens J
(2021)
Probabilistic Tsunami Hazard and Risk Analysis: A Review of Research Gaps
in Frontiers in Earth Science
Betcke T
(2021)
Designing a High-Performance Boundary Element Library With OpenCL and Numba
in Computing in Science & Engineering
Giles D
(2021)
Faster Than Real Time Tsunami Warning with Associated Hazard Uncertainties
in Frontiers in Earth Science
Gopinathan D
(2021)
Probabilistic quantification of tsunami current hazard using statistical emulation.
in Proceedings. Mathematical, physical, and engineering sciences
Hauptmann A
(2020)
Multi-Scale Learned Iterative Reconstruction.
in IEEE transactions on computational imaging
Lei T
(2021)
Pore-scale simulation of miscible viscous fingering with dissolution reaction in porous media
in Physics of Fluids
Ming D
(2021)
Linked Gaussian Process Emulation for Systems of Computer Models Using Matérn Kernels and Adaptive Design
in SIAM/ASA Journal on Uncertainty Quantification
Salmanidou D
(2021)
Probabilistic, high-resolution tsunami predictions in northern Cascadia by exploiting sequential design for efficient emulation
in Natural Hazards and Earth System Sciences
Wang G
(2021)
Lattice Boltzmann simulation of a water droplet penetrating a micropillar array in a microchannel
in Physics of Fluids
| Description | The outcomes so far have been reported in three international online workshops: 1. Workshop on "Software Engineering for Exascale", online, 14 - 15 July, 2020 2. Workshop on "Data Assimilation and Uncertainty Quantification at the exascale", 24 - 25 September, 2020 3. Workshop on "Towards Exascale Simulation of Integrated Engineering Systems at Extreme Scale", 21 - 22 January, 2021 |
| Exploitation Route | The feasibility studies and workshops so far will influence future training and development sprints, and software development activities. The work will contribute to the ongoing ExCALIBUR programmes, e.g. the cross-cutting research for exascale software and algorithms call. |
| Sectors | Aerospace, Defence and Marine,Energy,Environment,Manufacturing, including Industrial Biotechology,Pharmaceuticals and Medical Biotechnology,Transport |
| URL | https://excalibur-sle.github.io |
| Description | Work of the Working Group supported by this grant has fed into UKRI's strategies for the subsequent stages of the ExCALIBUR programme. Seven follow-on grants have been obtained by the Working Group members to further research and inform policy makers. |
| First Year Of Impact | 2022 |
| Sector | Digital/Communication/Information Technologies (including Software) |
| Impact Types | Policy & public services |
| Description | Advanced Quantification of Uncertainties In Fusion modelling at the Exascale with model order Reduction (AQUIFER) |
| Amount | £350,497 (GBP) |
| Funding ID | T/AW085/21 |
| Organisation | UK Atomic Energy Authority |
| Sector | Public |
| Country | United Kingdom |
| Start | 12/2021 |
| End | 03/2024 |
| Description | Integrated Simulation at the Exascale: coupling, synthesis and performance |
| Amount | £304,730 (GBP) |
| Funding ID | EP/W007460/1 |
| Organisation | Engineering and Physical Sciences Research Council (EPSRC) |
| Sector | Public |
| Country | United Kingdom |
| Start | 07/2021 |
| End | 07/2024 |
| Description | Software Environment for Actionable & VVUQ-evaluated Exascale Applications (SEAVIEW) |
| Amount | £728,469 (GBP) |
| Funding ID | EP/W007711/1 |
| Organisation | Engineering and Physical Sciences Research Council (EPSRC) |
| Sector | Public |
| Country | United Kingdom |
| Start | 07/2021 |
| End | 07/2024 |
| Description | Software Environment for Actionable & VVUQ-evaluated Exascale Applications (SEAVIEW) |
| Amount | £728,469 (GBP) |
| Funding ID | EP/W007711/1 |
| Organisation | Engineering and Physical Sciences Research Council (EPSRC) |
| Sector | Public |
| Country | United Kingdom |
| Start | 07/2021 |
| End | 07/2024 |
| Description | Steps towards Accurate Fusion modelling at the Exascale: Model Order Reduction (SAFE- MOR) |
| Amount | £53,706 (GBP) |
| Organisation | UK Atomic Energy Authority |
| Sector | Public |
| Country | United Kingdom |
| Start | 01/2021 |
| End | 07/2021 |
| Description | SysGenX: Composable software generation for system-level simulation at exascale |
| Amount | £979,027 (GBP) |
| Funding ID | EP/W026635/1 |
| Organisation | Engineering and Physical Sciences Research Council (EPSRC) |
| Sector | Public |
| Country | United Kingdom |
| Start | 12/2021 |
| End | 11/2024 |
| Description | The ExCALIBUR Hardware and Enabling Software (H&ES) Programme: FPGA Testbed |
| Amount | £254,397 (GBP) |
| Organisation | Science and Technologies Facilities Council (STFC) |
| Sector | Public |
| Country | United Kingdom |
| Start | 03/2021 |
| End | 03/2022 |
| Description | International Online Workshop on "Data Assimilation and Uncertainty Quantification at the Exascale" |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | The Workshop on "Data Assimilation and Uncertainty Quantification at the exascale" was held online during 24 - 25 September, 2020. It was attended by 180 people from academia, industry and UKRI reps. Talks addressed Uncertainty Quantification (UQ) covering the propagation of uncertainty from inputs to outputs through simulators, as well as inverse problems resulting from the calibration of models against observations. Also discussed was Data assimilation (DA), which synergizes computer simulations and real-world data, e.g. from weather prediction to hazard modelling, urban analytics and biological science, with observations used to update simulations in real time. With the interaction between forward simulations and information-driven methods, challenges of techniques for UQ and DA on exascale computing were identified, which called for close collaborations between RSEs, the research community and industry. |
| Year(s) Of Engagement Activity | 2020 |
| URL | https://excalibur-sle.github.io |
| Description | International Online Workshop on "Towards Exascale Simulation of Integrated Engineering Systems at Extreme Scale" |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | The Workshop on "Towards Exascale Simulation of Integrated Engineering Systems at Extreme Scales" was held online during 21 - 22 January, 2021 from a large audience from academia, industry and UKRI reps. In this workshop, leading experts in the field from China, Europe and USA as well as the UK showcased recent efforts in the development of modelling methodologies, computer algorithms, software interfaces, and software/hardware co-design in preparation for the arrival of exascale computing. Cutting-edge simulations of engineering systems at extreme time and length scales were demonstrated. Finally, existing gaps and remaining issues were identified, which called for further discussions and actions in the field. |
| Year(s) Of Engagement Activity | 2021 |
| URL | https://excalibur-sle.github.io |
| Description | Online International Workshop on "Software Engineering for Exascale" |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | The Workshop on "Software Engineering for Exascale" was held online during 14 - 15 July, 2020. It was attended by close to 200 people from academia, industry and UKRI. This workshop explored the challenges facing the scientific computing community, covering exascale readiness techniques such as performance portability, algorithmic fault tolerance, asynchronous programming, and improving scalability. The workshop included contributions from experts from international exascale initiatives such as ECP in the USA and EuroHPC. It also included input from leading vendors such as Cray/HPE, NVIDIA, AMD, Intel and Codeplay. Key questions addressed are: (i) how different simulation tools can be software-engineered for interoperability at scale; (ii) how future software engineering and new languages abstractions most effectively support the separation of concerns in development; and, (iii) how multi-physics simulations can optimally exploit heterogeneous hardware platforms. |
| Year(s) Of Engagement Activity | 2020 |
| URL | https://excalibur-sle.github.io |
| Description | Workshop IV: Inverse Problems and Optimisation |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Inverse problems are concerned with the recovery of the parameters of a forward model given observations of data that it describes. Such problems arise in almost all fields of science when details of a postulated model, such as maps of physical properties and/or their classification into identifiable objects, have to be determined from a set of observed data. The inverse problem topic is highly cross-disciplinary, both within mathematics, encompassing aspects of pure, applied and statistics, and across subjects, including physical sciences, engineering and biology to name only a few. Inverse problems increasingly consider mappings between solutions and data1in high numbers of dimensions (e.g. three in space plus time plus wavelength). Direct representations easily exceed existing computational and memory resources, and necessitate appropriate design of data structures and algorithms. In parallel, machine learning methods designed for "big data" problems are proving useful in developing data reduction approaches and representation of appropriate priors. This two-day workshop gathered together leading international experts with a worldwide audience, presenting the state of the art in the field. Some of the topics of this planned workshop include scheduling and optimising parallelism for multiple forward solves as part of a nonlinear inverse problem on exascale architectures; the combination of inference-based machine learning techniques and classical model based inverse problems at scale, and their often differing hardware requirements (e.g. GPU vs CPU); using exascale computing to include uncertainty in the formulation of inverse problems. |
| Year(s) Of Engagement Activity | 2021 |
| URL | https://excalibur-sle.github.io/workshop4/ |
| Description | Workshop V: Data Visualisation and Data Flows |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | The traditional workflow of the computational engineer or scientist: simulate, store in full, process later - will no longer be fit for purpose at exascale and a tighter coupling of simulation and post-processing codes will be required. Stakeholders at this workshop discussed the challenges of visualisation and data flow at exascale and topics such as: (i) limits of current workflows given roadmaps of future storage and I/O bandwidth; (ii) prescribed and automated in-situ data extraction; (iii) in-situ dimension reduction techniques; (iv) intelligent data compression; interactive analysis of large ensembles of simulations; and, (v) immersive visualisation using VR and AR. Invited speakers will include experts from exascale programs, open source initiatives, vendors and commercial visualization software companies. The workshop panels included practitioners from industry, government laboratories and academia, who are pioneering visualisation approaches at the current limits of both simulated and experimentally measured data flows. |
| Year(s) Of Engagement Activity | 2021 |
| URL | https://excalibur-sle.github.io/workshop4/ |