A scalable dynamical core for Next Generation Weather and Climate Prediction - Phase 2

Lead Research Organisation: University of Manchester
Department Name: Computer Science

Abstract

Historically, major improvements in the accuracy of numerical weather forecasts and climate simulations have come from the increased resolution enabled by the exponential growth in computer power. In order to achieve further gains in accuracy through further increases in resolution, it will be necessary to exploit the massively parallel computer architectures that are becoming available. However, current state-of-the-art operational algorithms are not expected to perform well beyond a few thousand processors: the grid structure of the traditional latitude-longitude grid means that interprocessor communication eventually but inevitably becomes a bottleneck.

The overall aim of the proposed project is to develop a new, three-dimensional, fully compressible dynamical core suitable for operational global and regional weather and climate prediction, as well as for research use, on massively parallel machines, and to demonstrate its accuracy, efficiency, and scalability. The accuracy should be comparable to that of existing state of the art algorithms. The algorithm must be efficient enough to run in the available operational time slots, and it must scale well on 100,000 to 1000,000 processors.

Phase 1 of this project (Feb 2011 - Jan 2013) addressed several of the basic scientific questions that underpin the development, including choice of quasi-uniform horizontal grid, choice of horizontal discretization, choice of transport scheme, time integration scheme, and some of the computer science aspects of the project. Several candidate approaches were tested and evaluated in a simplified two-dimensional fluid system (the Shallow Water Equations), and a small number of promising approaches were identified for further development in Phase 2.

Phase 2 of this project will build on the progress made in Phase 1 in order to develop a three-dimensional, fully compressible dynamical core. The work in Phase 2 falls broadly into three work packages:

* Vertical aspects. The stability and accuracy of the discretization depends crucially on the choice of vertical coordinate, the choice of thermodynamic variables predicted, and the vertical placement of variables relative to each other (`staggering'). It will also depend on the details of how, for example, the pressure gradient term is evaluated, especially near steep mountains, and how the vertical discretization couples with the horizontal discretization. Building on current understanding, candidate schemes will be formulated and tested.

* Code design and development. The code for the three-dimensional dynamical core will be based around a carefully designed software framework. The interface between the numerical discretization and its parallel implementation will be optimized, so that modifications to the former require minimal knowledge of the latter. The software framework will be highly flexible, so that it can easily accommodate future evolution of the dynamical core, such as changes in grid structure.

* Testing. The behaviour of complex numerical algorithms can be difficult to predict theoretically, even when individual components are well understood and tested. It will be vital, therefore, to test comprehensively the proposed formulations at the earliest opportunity, and revise if necessary. Early testing will focus on the shallow water formulation arising out of Phase 1 of the project, and on one-dimensional (column) and two-dimensional (vertical slice) prototypes of the vertical formulation. Testing of the three-dimensional formulation will begin as soon as code is available.

Planned Impact

Society benefits in numerous ways from accurate weather forecasts, via a wide range of weather-sensitive businesses and services (aviation, construction, energy, retail, ... etc.) as well as direct use by the public. Accurate forecasts of extreme weather events are particularly valuable in terms of minimizing risks to property as well as human life and health. Accurate predictions of climate change, particularly at a regional level, are essential for both climate-sensitive businesses and for policy makers, who must evaluate the costs and benefits (both economic and societal) of possible mitigation and adaptation measures. The proposed project will continue the drive towards more accurate weather and climate prediction by providing a key computational tool: a scalable atmospheric dynamical core that can take advantage of future massively parallel computing platforms to achieve higher resolution. Thus, the ultimate beneficiaries are the public, businesses, and policy makers who benefit from operational weather forecasts and climate predictions produced by the Met Office (as well as other users of the Met Office Unified Model around the world).

The immediate beneficiaries are the Met Office themselves. The proposal has been developed in close collaboration with the Met Office, and the project will involve a close partnership with the Met Office, building on the successful relationships and practices developed in Phase 1. UK academics and Met Office staff will be fully integrated in a single project team. There will be frequent project meetings, comprising quarterly plenary workshops interspersed with quarterly topical meetings, with day-to-day communication via a project email list and TWIKI. This close partnership will ensure that the project addresses the needs of the Met Office, is compatible with the other components of their operational system (physical parameterizations, data assimilation), and that the results pull through into their operational activities as rapidly and directly as possible.

The results of this project will be of great interest to the growing number of other groups around the world, both in operational centres and in and academic research, who, driven by the parallel scalability issue, are developing new atmospheric models. They will also be of wider interest to the computational fluid dynamics and high performance computing research communities. Results of this project will be published in the peer reviewed literature, and presented at relevant conferences and workshops, to reach the widest possible audience in both communities.

Publications

10 25 50
 
Description The research has contributed to the development of a new dynamical model of the atmosphere for potential use in the Met Office's weather and climate computer simulation model.

The major developments include a new software structure which separates into different layers the high-level scientific coding of an algorithm (Al), low level mesh-point-based kernel (K) computations and a middle, 'parallel system' (PSy) layer which translates the high level description into a set of low-level kernel calls. This middle layer also addresses exploiting parallelism in the algorithm for both shared memory and distributed memory computers. This approach to structuring code is called PSyKAl.

Research in the project is also demonstrating that the PSy layer can be generated from descriptions of the the algorithm and kernel layers, rather then requiring had coding. This approach has the potential to enable code to be ported to new computers and preserve an acceptable level of performance, a problem that is a major topic of research currently.
Exploitation Route The PSyKAl approach may well be adopted by the UK Met Office (who part fund the project) for their next generation weather and climate model. There is wider interest in the Earth System Modelling community worldwide and the developments in the project can be expected to influence future developments in that community. Future research will be able to explore how applicable the approach is to other engineering simulation communities, for example.
Sectors Digital/Communication/Information Technologies (including Software),Environment

 
Description The NGWCP academic project was joingly funded by NERC, STFC and the Met Office. The aim was to undertake research towards a new dynamical core for the Met Office. Since the academic project completed, the Met Office have continued to develop the prototype software and ideas that emerged from the project in the Met Office's LFRic project. STFC have also continued to support work on the Psyclone software developed primarily by themselves which is used in the LFRic project. Both the LFRic software and the Psyclone software are in the final stages of being released as open source software, enabling the wider research community to use them and contribute to their further development. Collaboration in three European projects has arisen based, in large part, on the developments in the two NERC-funded project with the Met Office. EuroEXA, developing a prototype Exascale computer; IS-ENES3, an infrastructures project with the Weather and Climate community; and ESiWACE2, a Centre of Excellence in Weather and Climate modelling.
First Year Of Impact 2016
Sector Digital/Communication/Information Technologies (including Software),Environment
Impact Types Societal

 
Description ESiWACE2, Excellence in the Simulation of Weather and Climate in Europe, phase 2. INFRAEDI-02-2018:HPC PPP - Centres of Excellence on HPC
Amount £8,034,389 (FJD)
Funding ID 823988 
Organisation European Commission 
Sector Public
Country European Union (EU)
Start 01/2019 
End 12/2022
 
Description EuroEXA, H2020-EU.1.2.2. - FET Proactive
Amount € 19,949,022 (EUR)
Funding ID Project ID: 754337 
Organisation European Commission H2020 
Sector Public
Country Belgium
Start 09/2017 
End 02/2021
 
Description IS-ENES3, Infrastructure for the European Network for Earth System Modelling - Phase 3. INFRAIA-01-2018-2019: Integrating Activities for Advanced Communities
Amount € 9,719,932 (EUR)
Funding ID 824084 
Organisation European Commission 
Sector Public
Country European Union (EU)
Start 01/2019 
End 12/2022