SysGenX: Composable software generation for system-level simulation at exascale
Lead Research Organisation:
UNIVERSITY OF CAMBRIDGE
Department Name: Engineering
Abstract
Systems modelled by partial differential equations (PDEs) are ubiquitous in science and engineering. They are used to model problems including structures, fluids, materials, electromagnetics, wave propagation and biological systems, and in areas as varied as aerospace, image processing, medical therapeutics and economics. PDEs comprise a forward model for predicting the response of a system, but are also a key component in the solution of inverse problems, for design optimisation, uncertainty quantification and data science applications, where the forward computation is repeated many times with different inputs.
The numerical simulation of complex systems modeled by PDEs is a challenging topic. It involves the choice of underlying equations, the selection of suitable numerical solvers, and implementation on specific hardware. Over the decades numerous software libraries have been developed to support this task. But adapting these libraries to the specific model and combining the various components in a low-level high-performance programming language requires a major development effort. This required effort has become significantly more challenging with the advent of heterogeneous mixed CPU/GPU devices on the path to exascale systems. Implementations need to be adapted for each individual device type in order to achieve good performance. As a consequence, developing new simulations at scale has become an ever more costly and time-intensive task.
In this project we propose a different simulation paradigm, based on the use of high-productivity languages such as Python to describe the problem, and automatic code generation and just-in-time compilation to translate the high-level formulations into high-performance exascale-ready code. Based on the experience with the component software libraries Firedrake, FEniCS and Bempp, the investigators will build a toolchain for complex exascale simulations of PDEs on unstructured grids, using state of the art finite element and boundary element technologies. The research will include mathematical and algorithmic underpinnings, concrete software development for automatic code generation of low-level CPU/GPU kernels, high-productivity language interfaces, and the application to 21st century exascale challenge problems in the areas of battery storage systems, net-zero flight, and high-frequency wave propagation.
The numerical simulation of complex systems modeled by PDEs is a challenging topic. It involves the choice of underlying equations, the selection of suitable numerical solvers, and implementation on specific hardware. Over the decades numerous software libraries have been developed to support this task. But adapting these libraries to the specific model and combining the various components in a low-level high-performance programming language requires a major development effort. This required effort has become significantly more challenging with the advent of heterogeneous mixed CPU/GPU devices on the path to exascale systems. Implementations need to be adapted for each individual device type in order to achieve good performance. As a consequence, developing new simulations at scale has become an ever more costly and time-intensive task.
In this project we propose a different simulation paradigm, based on the use of high-productivity languages such as Python to describe the problem, and automatic code generation and just-in-time compilation to translate the high-level formulations into high-performance exascale-ready code. Based on the experience with the component software libraries Firedrake, FEniCS and Bempp, the investigators will build a toolchain for complex exascale simulations of PDEs on unstructured grids, using state of the art finite element and boundary element technologies. The research will include mathematical and algorithmic underpinnings, concrete software development for automatic code generation of low-level CPU/GPU kernels, high-productivity language interfaces, and the application to 21st century exascale challenge problems in the areas of battery storage systems, net-zero flight, and high-frequency wave propagation.
Organisations
- UNIVERSITY OF CAMBRIDGE (Lead Research Organisation)
- Codeplay Software Ltd (Project Partner)
- University of Colorado Boulder (Project Partner)
- University of Muenster (Munster) (Project Partner)
- University of Buffalo (Project Partner)
- Lawrence Livermore National Laboratory (Project Partner)
- Turbostream Ltd (Project Partner)
- nVIDIA (Project Partner)
- UNITED KINGDOM ATOMIC ENERGY AUTHORITY (Project Partner)
Publications
Scroggs M
(2022)
Construction of Arbitrary Order Finite Element Degree-of-Freedom Maps on Polygonal and Polyhedral Cell Meshes
in ACM Transactions on Mathematical Software
Dean J
(2023)
Design and analysis of an exactly divergence-free hybridised discontinuous Galerkin method for incompressible flows on meshes with quadrilateral cells
in Computer Methods in Applied Mechanics and Engineering
Scroggs M
(2022)
Basix: a runtime finite element basis evaluation library
in Journal of Open Source Software
Richardson C
(2025)
An efficient multigrid solver for finite element methods on multi-GPU systems
in Procedia Computer Science
Baratta I
(2025)
DOLFINx: The next generation FEniCS problem solving environment
| Description | This project has developed high-performance prototype solvers for partial differential equations using massively parallel GPU supercomputers. The techniques and open-source software created have been demonstrated to achieve a high fraction of the theoretical peak performance of modern GPU architectures, and simulations have been run on some of the world's largest supercomputers. |
| Exploitation Route | Software outputs of the project are open-source, and can be freely used and further developed by others. |
| Sectors | Aerospace Defence and Marine Digital/Communication/Information Technologies (including Software) |
| Description | Methods and software developed in this projects have: - Formed the basis of an application benchmark that is being used in the procurement on the Next National Supercomputing Service, a £750M project being delivered by UKRI. - Open-source software developed in the projects is being used by a range of companies, including Rolls-Royce and Turbostream. |
| First Year Of Impact | 2026 |
| Sector | Aerospace, Defence and Marine,Digital/Communication/Information Technologies (including Software) |
| Impact Types | Economic |
| Title | Software, Dataset, and Techreport: Mixed-precision finite element kernels and assembly: Rounding error analysis and hardware acceleration |
| Description | This upload contains a techreport titled "Mixed-precision finite element kernels and assembly: Rounding error analysis and hardware acceleration" together with the software (with documentation) and dataset generating the results. The software is also available on GitHub at https://github.com/croci/mpfem-paper-experiments-2024/ . The GitHub version may be updated in the future. This upload corresponds to commit number 8506dd368b84655201c8c72b1307239b9b4e43fd . See README.md file for installation instructions. The manuscript is also available on the arXiv: https://arxiv.org/abs/2410.12614. |
| Type Of Material | Database/Collection of data |
| Year Produced | 2024 |
| Provided To Others? | Yes |
| URL | https://zenodo.org/doi/10.5281/zenodo.13941628 |
| Title | Software, Dataset, and Techreport: Mixed-precision finite element kernels and assembly: Rounding error analysis and hardware acceleration |
| Description | This upload contains a techreport titled "Mixed-precision finite element kernels and assembly: Rounding error analysis and hardware acceleration" together with the software (with documentation) and dataset generating the results. The software is also available on GitHub at https://github.com/croci/mpfem-paper-experiments-2024/ . The GitHub version may be updated in the future. This upload corresponds to commit number 8506dd368b84655201c8c72b1307239b9b4e43fd . See README.md file for installation instructions. The manuscript is also available on the arXiv: https://arxiv.org/abs/2410.12614. |
| Type Of Material | Database/Collection of data |
| Year Produced | 2024 |
| Provided To Others? | Yes |
| URL | https://zenodo.org/doi/10.5281/zenodo.13941629 |
