Parallel Paradigms for Numerical Weather Prediction
Lead Research Organisation:
Imperial College London
Department Name: Mathematics
Abstract
Weather forecasts and climate simulations require dedicated high
performance supercomputers to run. Advances in the power of
supercomputers bring the possibility of simulating the atmosphere at
higher resolution (i.e. with more detail) without having to wait
longer for the answer. It has been consistently shown that increasing
the resolution of atmosphere models results in more accurate weather
forecasts and climate simulations. However, getting models that can
make full use of state-of-the-art supercomputers is very challenging.
The Met Office is in the process of installing a new Cray XC40
supercomputer which which will deliver 16 petaflops (16 quadrillion
arithmetic operations per second)
peak processing
power by using 4800000 individual processors computing
together at the same time (in parallel). In the next few decades
supercomputers are expected to deliver more and more computing power,
by using more and more processors. The main thing that slows down
computations on these massively parallel supercomputers is
communicating data between processors. Unfortunately, the physics of the
atmosphere means that the weather in one location is intrinsically
linked with the weather at all other locations on the globe; this
means that a lot of data communication between processors is required.
Scientists who develop atmosphere models are currently grappling with
the fact that we are close to the limit of what is possible in terms
of resolution and simulation speed, due to the communication
requirements of the mathematical algorithms that are used to solve the
equations that predict how the weather evolves in time. At the moment,
these algorithms use geographic parallelism: the globe is divided up
into overlapping pieces and each piece is given to a different
processor, which must communicate data to processors that share
geographic locations on the overlaps. To speed up a model, we need to
use more and more processors on smaller and smaller regions. The
speed-up is eventually limited when there are so many overlapping regions
that all of the globe is
covered by overlaps, and the model spends all of the time
communicating.
This means that it is time to invent new mathematical algorithms that
can make better use of the parallel computer. In this project we will develop
algorithms that are time-parallel as well as
geographic-parallel. Instead of advancing the forecast of the model
forwards step by step in time, these methods produce several different
estimates of the weather at the next step, before combining them
together to make a more accurate solution. Each of these different
estimates can be independently calculated, which introduces additional
parallel computation into the model.
This project is in close partnership with the Met Office. If
successful, these algorithms will lead to faster and higher resolution
weather forecast and climate prediction models at the Met Office,
leading to more accurate forecasts for government, industry and the
general public. The Met Office provides forecasts for customers across
the transport sector, particularly for aviation planning (so that
aeroplanes can avoid headwinds and make use of tailwinds) and
predictions of the motion of volcanic ash clouds. It also provides
forecasts for retail and leisure, insurers, the Ministry of Defence, and the
Environment Agency (including flood forecasting). More accurate
forecasts will allow all of these business organisations to plan further
into the future, avoiding risks and unnecessary costs.
performance supercomputers to run. Advances in the power of
supercomputers bring the possibility of simulating the atmosphere at
higher resolution (i.e. with more detail) without having to wait
longer for the answer. It has been consistently shown that increasing
the resolution of atmosphere models results in more accurate weather
forecasts and climate simulations. However, getting models that can
make full use of state-of-the-art supercomputers is very challenging.
The Met Office is in the process of installing a new Cray XC40
supercomputer which which will deliver 16 petaflops (16 quadrillion
arithmetic operations per second)
peak processing
power by using 4800000 individual processors computing
together at the same time (in parallel). In the next few decades
supercomputers are expected to deliver more and more computing power,
by using more and more processors. The main thing that slows down
computations on these massively parallel supercomputers is
communicating data between processors. Unfortunately, the physics of the
atmosphere means that the weather in one location is intrinsically
linked with the weather at all other locations on the globe; this
means that a lot of data communication between processors is required.
Scientists who develop atmosphere models are currently grappling with
the fact that we are close to the limit of what is possible in terms
of resolution and simulation speed, due to the communication
requirements of the mathematical algorithms that are used to solve the
equations that predict how the weather evolves in time. At the moment,
these algorithms use geographic parallelism: the globe is divided up
into overlapping pieces and each piece is given to a different
processor, which must communicate data to processors that share
geographic locations on the overlaps. To speed up a model, we need to
use more and more processors on smaller and smaller regions. The
speed-up is eventually limited when there are so many overlapping regions
that all of the globe is
covered by overlaps, and the model spends all of the time
communicating.
This means that it is time to invent new mathematical algorithms that
can make better use of the parallel computer. In this project we will develop
algorithms that are time-parallel as well as
geographic-parallel. Instead of advancing the forecast of the model
forwards step by step in time, these methods produce several different
estimates of the weather at the next step, before combining them
together to make a more accurate solution. Each of these different
estimates can be independently calculated, which introduces additional
parallel computation into the model.
This project is in close partnership with the Met Office. If
successful, these algorithms will lead to faster and higher resolution
weather forecast and climate prediction models at the Met Office,
leading to more accurate forecasts for government, industry and the
general public. The Met Office provides forecasts for customers across
the transport sector, particularly for aviation planning (so that
aeroplanes can avoid headwinds and make use of tailwinds) and
predictions of the motion of volcanic ash clouds. It also provides
forecasts for retail and leisure, insurers, the Ministry of Defence, and the
Environment Agency (including flood forecasting). More accurate
forecasts will allow all of these business organisations to plan further
into the future, avoiding risks and unnecessary costs.
Planned Impact
The goal of this project is to design mathematical algorithms that will allow numerical weather prediction models to run efficiently on the next generation of massively parallel supercomputers. This means that operational weather forecasting centres such as the Met Office will be able to continue their programme of increasing model resolution, which leads to increased forecasting skill. It also means that climate scientists can perform climate simulations with higher accuracy, or get their results more quickly.
An increase in forecasting skill equates to reduced (or more precisely estimated) uncertainty about future weather. The Met Office has a large range of customers, providing forecasts for
aviation and flight planning, as well as for the marine, road and rail
industries. It also provides forecasts for smoke clouds generated by
volcanoes, advising air traffic control, and forecasts for mining,
building and construction, retail, finance and utilities, and the
defence sector. All of these industries would benefit from extending the window of high forecasting skill, by providing more accurate forecasts and quantification of their uncertainty further into the future, allowing for planning and investment. The ability to run climate models faster or at higher resolution will also extend the depth of science questions that can be explored by climate scientists, and hence benefit end users including the insurance industry, policymakers and the general public.
An increase in forecasting skill equates to reduced (or more precisely estimated) uncertainty about future weather. The Met Office has a large range of customers, providing forecasts for
aviation and flight planning, as well as for the marine, road and rail
industries. It also provides forecasts for smoke clouds generated by
volcanoes, advising air traffic control, and forecasts for mining,
building and construction, retail, finance and utilities, and the
defence sector. All of these industries would benefit from extending the window of high forecasting skill, by providing more accurate forecasts and quantification of their uncertainty further into the future, allowing for planning and investment. The ability to run climate models faster or at higher resolution will also extend the depth of science questions that can be explored by climate scientists, and hence benefit end users including the insurance industry, policymakers and the general public.
Publications
Bauer W
(2018)
Energy-enstrophy conserving compatible finite element schemes for the rotating shallow water equations with slip boundary conditions
in Journal of Computational Physics
Bauer W
(2022)
Higher Order Phase Averaging for Highly Oscillatory Systems
in Multiscale Modeling & Simulation
Bendall T
(2020)
A compatible finite-element discretisation for the moist compressible Euler equations
in Quarterly Journal of the Royal Meteorological Society
Bendall T
(2019)
The 'recovered space' advection scheme for lowest-order compatible finite element methods
in Journal of Computational Physics
Brecht R
(2021)
Rotating Shallow Water Flow Under Location Uncertainty With a Structure-Preserving Discretization
in Journal of Advances in Modeling Earth Systems
Brecht R
(2021)
Selective decay for the rotating shallow-water equations with a structure-preserving discretization
in Physics of Fluids
Clarke A
(2020)
Parallel-in-time integration of kinematic dynamos
in Journal of Computational Physics: X
Clarke A
(2020)
Performance of parallel-in-time integration for Rayleigh Bénard convection
in Computing and Visualization in Science
Cotter C
(2023)
Singular solutions of the r-Camassa-Holm equation
in Nonlinearity
Description | We have developed an efficient solver framework for a new type of "phase-averaged" weather model that allows to take very large timesteps through parallelisation over averages. We have also developed a new framework for time-parallel atmosphere models based on the Paradiag idea. This has been implemented as a software library called asQ (published on Github). |
Exploitation Route | When combined with other approaches, the averaging framework can be used to make more efficient use of next generation parallel computers for operational weather forecasting. The asQ software library is underpinning our newly funded work in the Excalibur project. |
Sectors | Digital/Communication/Information Technologies (including Software) Environment |
URL | https://github.com/colinjcotter/asQ |
Description | Algorithms were designed following the research in this project which are now being used to demonstrate time-parallel simulation for the Met Office and UKAEA. This may lead to the use of time-parallel algorithms at those labs in the future, to simulate weather and fusion power. |
First Year Of Impact | 2023 |
Sector | Energy,Environment |
Impact Types | Policy & public services |
Description | Next generation particle filters for stochastic partial differential equations |
Amount | £304,744 (GBP) |
Funding ID | EP/W016125/1 |
Organisation | Engineering and Physical Sciences Research Council (EPSRC) |
Sector | Public |
Country | United Kingdom |
Start | 06/2022 |
End | 12/2024 |
Description | SPF EX20-8 Exposing Parallelism: Parallelin-Time (DN517492) |
Amount | £1,152,298 (GBP) |
Funding ID | SPF ExCALIBUR: EX20-8 |
Organisation | Meteorological Office UK |
Sector | Academic/University |
Country | United Kingdom |
Start | 03/2021 |
End | 03/2024 |
Description | TIME-X: Time parallelization for eXascale computing |
Amount | € 302,425,375 (EUR) |
Funding ID | 955701 |
Organisation | European Union |
Sector | Public |
Country | European Union (EU) |
Start | 03/2021 |
End | 03/2024 |
Title | asQ |
Description | A software modelling system for building time-parallel implementations of weather and climate models. |
Type Of Material | Computer model/algorithm |
Year Produced | 2021 |
Provided To Others? | Yes |
Impact | This has led to further funding from the Met Office on time parallel algorithms. |
URL | https://github.com/colinjcotter/asQ |