Uncertainty Quantification at the Exascale (EXA-UQ)
Lead Research Organisation:
UNIVERSITY OF EXETER
Department Name: Mathematics
Abstract
Exascale computing offers the prospect of running numerical models, for example of nuclear fusion and the climate, at unprecedented resolution and fidelity, but such models are still subject to uncertainty and we need to able to quantify such uncertainties (and for example use data on model outputs to calibrate the model inputs). Exascale computing comes at a cost. We will never be able to run huge ensembles go models on Exascale computers. Naive methods, such as Monte Carlo where we simply sample from the probability distribution of the model inputs, run a huge ensemble of models and produce a sample from the output distribution, are not going to be feasible. We need to develop uncertainty quantification methodology that allows us to efficiently, and effectively, perform sensitivity and uncertainty calculations with the minimum number of exascale model runs.
Our methods are based on the idea of an emulator. An emulator is a statistical approximation linking model inputs and outputs in a fast non-linear way. It also includes a measure of its own uncertainty so we know how well it is approximating the original numerical model. Our emulators are based on Gaussian processes. Normally we would run a designed experiment and use these results to train the emulator. Because of the cost of exascale computing we use a hierarchy of models from fast, low fidelity versions through higher fidelity more computationally expensive ones to the very expensive, very high fidelity one at the apex of the hierarchy. Building a joint emulator for all the models in the hierarchy allows us to gain strength from the low fidelity ones to emulate the exascale models. Although such ideas have been around for a number of years they have not been exploited much for very large models.
We will expand on the existing theory on a number of new ways. First we will look at the problem of design. To exploit the hierarchy to its fullest extent we need an experimental design that allocates model runs to the correct layer of the model hierarchy. We will extend existing sequential design methodology to work with hierarchies of model, not only finding the optimal next set of inputs for running the model but also which level it should be run in. We will also ensure that the sequential design is 'batch' sequential, allowing us to run ensembles rather than waiting for each run to return answers.
Because the inputs and outputs of exascale models are often fields of correlated values we will develop methods for handling such high dimensional inputs and outputs and how to relate them to other levels of the hierarchy.
Finally we will investigate whether AI methods other than Gaussian processes can be used to build efficient emulators.
Our methods are based on the idea of an emulator. An emulator is a statistical approximation linking model inputs and outputs in a fast non-linear way. It also includes a measure of its own uncertainty so we know how well it is approximating the original numerical model. Our emulators are based on Gaussian processes. Normally we would run a designed experiment and use these results to train the emulator. Because of the cost of exascale computing we use a hierarchy of models from fast, low fidelity versions through higher fidelity more computationally expensive ones to the very expensive, very high fidelity one at the apex of the hierarchy. Building a joint emulator for all the models in the hierarchy allows us to gain strength from the low fidelity ones to emulate the exascale models. Although such ideas have been around for a number of years they have not been exploited much for very large models.
We will expand on the existing theory on a number of new ways. First we will look at the problem of design. To exploit the hierarchy to its fullest extent we need an experimental design that allocates model runs to the correct layer of the model hierarchy. We will extend existing sequential design methodology to work with hierarchies of model, not only finding the optimal next set of inputs for running the model but also which level it should be run in. We will also ensure that the sequential design is 'batch' sequential, allowing us to run ensembles rather than waiting for each run to return answers.
Because the inputs and outputs of exascale models are often fields of correlated values we will develop methods for handling such high dimensional inputs and outputs and how to relate them to other levels of the hierarchy.
Finally we will investigate whether AI methods other than Gaussian processes can be used to build efficient emulators.
Publications
Green M
(2024)
Flight-ready Electrical Capacitance Tomography SMARTTS tank for use with cryogenics
in Experimental Thermal and Fluid Science
Green M
(2024)
NekMesh: An open-source high-order mesh generation framework
in Computer Physics Communications
Mohammadi H
(2022)
Cross-Validation--based Adaptive Sampling for Gaussian Process Models
in SIAM/ASA Journal on Uncertainty Quantification
Young J
(2022)
Social sensing of flood impacts in India: A case study of Kerala 2018
in International Journal of Disaster Risk Reduction
Young JC
(2024)
CIDER: Context-sensitive polarity measurement for short-form text.
in PloS one
| Description | We have developed sequential experimental design techniques that will allow model runs carried out on less expensive computers to feed into the uncertainty quantification on the expensive exascale machine. We have also developed a new Bayesian method of using these cheaper runs to carry out the UQ on the Exascle machine |
| Exploitation Route | We are publicising our results through a series of outreach events and through an easy use python package |
| Sectors | Aerospace Defence and Marine Chemicals Construction Energy Environment Healthcare Manufacturing including Industrial Biotechology Pharmaceuticals and Medical Biotechnology |
| Title | EXA-UQ Toolbox |
| Description | When running computer experiments on the largest high performance computers it is not possible to run enough simulations to carry out uncertainty quantification, even if we use surrogate models such as Gaussian process emulators. One solution to this problem is to use hierarchies of computer comers where we have very expensive version with high fidelity that need very powerful machines and simpler versions that have lower fidelity but can run on less computationally expensive machines at lower fidelity. Inferences about the uncertainty on the highest fidelity model are made from a combination of the whole hierarchy.. The Exa-UQ toolbox provides python code that allows such experiments to be designed in a new sequential way, where we use the results of previous model runs to decide where to run the nekton (both its position in input space and its ;even in the hierarchy. We have also implemented a new method (named BAYHEM) for carrying outage inference in a new efficient non-linear way. |
| Type Of Material | Improvements to research infrastructure |
| Year Produced | 2025 |
| Provided To Others? | Yes |
| Impact | We haven't yet used the tool in any practical examples butexpectto soon |
| URL | https://github.com/EXA-UQ/EXAUQ-Toolbox |
| Title | EXA-UQ/EXAUQ-Toolbox: v0.2.0 |
| Description | What's Changed Implement Multi-Interface Initialisation and Management in CLI by @mbjohns in https://github.com/EXA-UQ/EXAUQ-Toolbox/pull/282 Iss377 update pyprojecttoml to pull down latest version by @HarryTWhite in https://github.com/EXA-UQ/EXAUQ-Toolbox/pull/378 Update test-package.yml to run poetry lock --no-update by @HarryTWhite in https://github.com/EXA-UQ/EXAUQ-Toolbox/pull/361 Add -v / --version argument to exauq main by @HarryTWhite in https://github.com/EXA-UQ/EXAUQ-Toolbox/pull/379 Fix empty input in TrainingDatum and add test by @HarryTWhite in https://github.com/EXA-UQ/EXAUQ-Toolbox/pull/360 Update Python package dependencies by @github-actions in https://github.com/EXA-UQ/EXAUQ-Toolbox/pull/381 Iss287 add seq from array class method to input by @HarryTWhite in https://github.com/EXA-UQ/EXAUQ-Toolbox/pull/362 Update Python package dependencies by @github-actions in https://github.com/EXA-UQ/EXAUQ-Toolbox/pull/382 Fix CI workflow: Remove deprecated --no-update flag from Poetry lock command by @mbjohns in https://github.com/EXA-UQ/EXAUQ-Toolbox/pull/385 Update Python package dependencies by @github-actions in https://github.com/EXA-UQ/EXAUQ-Toolbox/pull/383 Toolbox code and documentation tidy by @HarryTWhite in https://github.com/EXA-UQ/EXAUQ-Toolbox/pull/321 Update Python package dependencies by @github-actions in https://github.com/EXA-UQ/EXAUQ-Toolbox/pull/389 Update README to Reflect Removal of poetry shell in Poetry 2.0 by @mbjohns in https://github.com/EXA-UQ/EXAUQ-Toolbox/pull/387 Release v0.2.0 by @mbjohns in https://github.com/EXA-UQ/EXAUQ-Toolbox/pull/396 Full Changelog: https://github.com/EXA-UQ/EXAUQ-Toolbox/compare/v0.1.1...v0.2.0 |
| Type Of Technology | Software |
| Year Produced | 2025 |
| Open Source License? | Yes |
| Impact | None yet |
| URL | https://zenodo.org/doi/10.5281/zenodo.15005643 |
| Description | AI & Data Science for Marine Environments |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | Local |
| Primary Audience | Professional Practitioners |
| Results and Impact | Workshop on these of AI and data science marine science |
| Year(s) Of Engagement Activity | 2023 |
| Description | Interview for Chinese television |
| Form Of Engagement Activity | A press release, press conference or response to a media enquiry/interview |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Public/other audiences |
| Results and Impact | An interview with Chinese television on the future ofAI |
| Year(s) Of Engagement Activity | 2024 |
| Description | Meeting to engage with computational Social Scientists |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | To investigate possible collaborations and use our methodology by computational social scientists |
| Year(s) Of Engagement Activity | 2024 |
| Description | Panelsession on the future ofAI |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | Local |
| Primary Audience | Public/other audiences |
| Results and Impact | A panel session for the public on the future of Artificial Intelligence.The event sold out and sparked lively discussions. |
| Year(s) Of Engagement Activity | 2022 |
| Description | Public engagement even with Agile Rabbit |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | Local |
| Primary Audience | Public/other audiences |
| Results and Impact | We ran a stand in a pop-up shop in Exeter |
| Year(s) Of Engagement Activity | 2022 |
| Description | South West Business Council Event on AI |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | Regional |
| Primary Audience | Industry/Business |
| Results and Impact | We had a stand explaining our research at a business event on AI |
| Year(s) Of Engagement Activity | 2023 |
| Description | Stand at AIUK2023 |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | We had a stand with some live demonstrations.These were used to initiate conversations about our research with AI/UK participants. |
| Year(s) Of Engagement Activity | 2023 |
| Description | Workshop on mathematics and Machine Learning |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | Local |
| Primary Audience | Professional Practitioners |
| Results and Impact | A workshop to investigate the uses of machine learning in mathematics and to a lesser extent thereof mathematics in ML |
| Year(s) Of Engagement Activity | 2023 |
| Description | Workshop with Rothamsted Research |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | Local |
| Primary Audience | Professional Practitioners |
| Results and Impact | Workshop to explore joint work with agricultural scientists |
| Year(s) Of Engagement Activity | 2023 |
