A scalable dynamical core for Next Generation Weather and Climate Prediction - Phase 2

Lead Research Organisation: University of Reading
Department Name: Meteorology


Historically, major improvements in the accuracy of numerical weather forecasts and climate simulations have come from the increased resolution enabled by the exponential growth in computer power. In order to achieve further gains in accuracy through further increases in resolution, it will be necessary to exploit the massively parallel computer architectures that are becoming available. However, current state-of-the-art operational algorithms are not expected to perform well beyond a few thousand processors: the grid structure of the traditional latitude-longitude grid means that interprocessor communication eventually but inevitably becomes a bottleneck.

The overall aim of the proposed project is to develop a new, three-dimensional, fully compressible dynamical core suitable for operational global and regional weather and climate prediction, as well as for research use, on massively parallel machines, and to demonstrate its accuracy, efficiency, and scalability. The accuracy should be comparable to that of existing state of the art algorithms. The algorithm must be efficient enough to run in the available operational time slots, and it must scale well on 100,000 to 1000,000 processors.

Phase 1 of this project (Feb 2011 - Jan 2013) addressed several of the basic scientific questions that underpin the development, including choice of quasi-uniform horizontal grid, choice of horizontal discretization, choice of transport scheme, time integration scheme, and some of the computer science aspects of the project. Several candidate approaches were tested and evaluated in a simplified two-dimensional fluid system (the Shallow Water Equations), and a small number of promising approaches were identified for further development in Phase 2.

Phase 2 of this project will build on the progress made in Phase 1 in order to develop a three-dimensional, fully compressible dynamical core. The work in Phase 2 falls broadly into three work packages:

* Vertical aspects. The stability and accuracy of the discretization depends crucially on the choice of vertical coordinate, the choice of thermodynamic variables predicted, and the vertical placement of variables relative to each other (`staggering'). It will also depend on the details of how, for example, the pressure gradient term is evaluated, especially near steep mountains, and how the vertical discretization couples with the horizontal discretization. Building on current understanding, candidate schemes will be formulated and tested.

* Code design and development. The code for the three-dimensional dynamical core will be based around a carefully designed software framework. The interface between the numerical discretization and its parallel implementation will be optimized, so that modifications to the former require minimal knowledge of the latter. The software framework will be highly flexible, so that it can easily accommodate future evolution of the dynamical core, such as changes in grid structure.

* Testing. The behaviour of complex numerical algorithms can be difficult to predict theoretically, even when individual components are well understood and tested. It will be vital, therefore, to test comprehensively the proposed formulations at the earliest opportunity, and revise if necessary. Early testing will focus on the shallow water formulation arising out of Phase 1 of the project, and on one-dimensional (column) and two-dimensional (vertical slice) prototypes of the vertical formulation. Testing of the three-dimensional formulation will begin as soon as code is available.

Planned Impact

see lead
Description I have been developing numerical methods for transporting pollutants and other atmospheric constituents that are stable and accurate even when a large time-step are used and even on quasi-uniform grids of the sphere. The Met Office is moving a way from the highly non-uniform latitude-longitude grid to a quasi-uniform grid such as the cubed sphere. Therefore it is not possible to use the same kind of transport scheme. The question is, should we use a one-dimensional scheme applied in every direction separately or a multi-dimensional scheme that does not rely on grid points being aligned in particular directions. With an MSc student, we have developed a new transport scheme that does not rely on grid alignment. However we have found that the classic schemes that do rely on grid alignment, perform extremely well even when grids are not aligned.

We have also compared the accuracy of different techniques for taking long, stable time-steps. Implicit time-stepping is usually thought to suffer from large errors when using large time-steps while semi-Lagrangian schemes are thought to be more accurate. In contrast, we have found that the more Lagrangian type of transport scheme is unstable when long time-steps are taken on distorted grids while the implicit time-stepping retains stability.
Exploitation Route Continued discussions with the Met Office and with ECMWF
Sectors Aerospace, Defence and Marine,Environment

URL https://arxiv.org/abs/1701.06907
Description We have identified severe problems with one of the numerical methods that the Met Office plan to use for their next generation dynamical core and have proposed an alternative. We have identified problems with using one-dimensional schemes in each direction separately for models which do not have grid-lines aligned at right angles to each other. A new type of transport scheme has been developed for use in the next generation Met Office model which has been compared with classic transport schemes. It is still undecided which type of the transport scheme the Met Office will use but our results will help to inform their decision.
First Year Of Impact 2014
Sector Aerospace, Defence and Marine
Impact Types Policy & public services

Description Transport scheme in next generation Met Office model
Geographic Reach National 
Policy Influence Type Participation in a advisory committee
URL https://arxiv.org/abs/1701.06907
Title AtmosFOAM 
Description A suite of numerical models and methods for atmospheric modelling using emerging numerical techniques 
Type Of Material Computer model/algorithm 
Year Produced 2014 
Provided To Others? Yes  
Impact Papers and collaborative code development with students 
URL https://github.com/AtmosFOAM/
Description MSc project on transport schemes with the Met Office 
Organisation Meteorological Office UK
Country United Kingdom 
Sector Public 
PI Contribution Tested various transport schemes which could be suitable for the next Met Office forecasting model.
Collaborator Contribution Met Office staff jointly supervised an MSc dissertation on numerical methods suitable for the next Met Office forecasting model.
Impact Publication submitted: https://arxiv.org/abs/1701.06907
Start Year 2015
Description Met Office/NERC/STFC Gung Ho project 
Organisation Meteorological Office UK
Country United Kingdom 
Sector Public 
PI Contribution Under this grant, work is ongoing on the Gung Ho project joint between the Met Office, academics from six Universities and STFC to design and build the dynamical core of a new weather and climate forecasting model. In particular, the contributions by Hilary Weller to this project have been joint with Nigel Wood at the Met Office, Sarah-Jane Lock at Leeds (now at ECMWF), John Thuburn at Exeter and Colin Cotter at Imperial College.
Collaborator Contribution A new dynamical core of the global atmosphere is being developed
Impact Lock, S.-J., Wood, N. and Weller, H. (2014) Numerical analyses of Runge-Kutta implicit-explicit schemes for horizontally explicit, vertically implicit solutions of atmospheric models. Quarterly Journal of the Royal Meteorological Society. ISSN 1477-870X doi: 10.1002/qj.2246 Weller, H. and Shahrokhi, A. (2014) Curl free pressure gradients over orography in a solution of the fully compressible Euler equations with implicit treatment of acoustic and gravity waves. Monthly Weather Review. ISSN 1520-0493 doi: 10.1175/MWR-D-14-00054.1 Weller, H. (2013) Non-orthogonal version of the arbitrary polygonal C-grid and a new diamond grid. Geoscientific Model Development, 7. pp. 779-797. ISSN 1991-959X doi: 10.5194/gmd-7-779-2014 Weller, H., Lock, S.-J. and Wood, N. (2013) Runge-Kutta IMEX schemes for the Horizontally Explicit/Vertically Implicit (HEVI) solution of wave equations. Journal of Computational Physics, 252. pp. 365-381. ISSN 0021-9991 doi: 10.1016/j.jcp.2013.06.025 Weller, H. (2012) Controlling the computational modes of the arbitrarily structured C-grid. Monthly Weather Review, 140 (10). pp. 3220-3234. ISSN 1520-0493 doi: 10.1175/MWR-D-11-00221.1 Weller, H., Thuburn, J. and Cotter, C. J. (2012) Computational modes and grid imprinting on five quasi-uniform spherical C-grids. Monthly Weather Review, 140 (8). pp. 2734-2755. ISSN 1520-0493 doi: 10.1175/MWR-D-11-00193.1
Start Year 2011
Title AtmosFOAM 
Description A set of library routines and applications for simulating the atmosphere using arbitrary meshes 
Type Of Technology Software 
Year Produced 2015 
Open Source License? Yes  
Impact This is a research tool and has enabled the research for all of my publications. 
URL https://github.com/AtmosFOAM
Description Dynamical Core Model Intercomparison Project (DCMIP) 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Series of lectures at the Dynamical Core Model Intercomparison Project workshop in order to train post-graduates and weather forecast model developers in the latest modelling techniques.
Year(s) Of Engagement Activity 2016
URL https://www.earthsystemcog.org/projects/dcmip-2016/
Description Plenary talk at SIAM student chapter 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Postgraduate students
Results and Impact The talk on "New mathematical techniques for weather and climate prediction" sparked questions and interest from the students and informed students about the mathematical challenges of weather and climate prediction

Year(s) Of Engagement Activity 2014
URL http://www.reading.ac.uk/maths/siamstudentchapter/conference.html