AtmosFOAM parallel scaling on HECToR and New Test Cases which expose Grid Scale Oscillations

Some of the best weather and climate forecasting models of the past few decades appear not to work well on the computer architectures of the coming decades. Further increases in computer power need to be accompanied by reduced electricity consumption and this, along with hardware cost, means that future generations of high performance computers will consist of a tens or hundreds of thousands of cheaper, less powerful processing cores, such as are used by computer games. Forecasting models need to be designed differently so that the computation can be distributed efficiently over so many less powerful cores. In designing new forecasting models that will perform well on next generation computers it is essential to compare different aspects of the model independently and it is essential to be able to test simplified model versions rigorously before the entire model is assembled. These two aspects will be covered separately in this project: (1) It is proposed to further develop a simplified model of the global atmosphere: AtmosFOAM, to run on 10,000 processing cores of the UK high performance computer, HECToR. AtmosFOAM is written using the OpenFOAM software library which has already been run on HECToR. Other models written using OpenFOAM have been shown to scale well to 1,000 or more cores on HECToR. It is aimed to reach a horizontal spatial resolution of 15km with 50 vertical levels meaning around 150 million computational points with 10 to 15 thousand points on each core. This problem size should be large enough to scale to 10,000 cores. Two of the bottlenecks to parallel scaling (identified by the Met Office) are semi-implicit time stepping which allows longer time steps but requires synchronised, global communication across all processing cores, and the smoothing of the solutions near the north and south poles where the lines of constant longitude converge. Semi-implicit time stepping is important because it means that the time step does not have to be short enough to resolve sound waves. However if explicit time stepping is used instead in the horizontal, less processor communication is needed but smaller time steps are needed. At some point it will become beneficial to go back to the modelling techniques of the 1980s and use time stepping which is explicit in the horizontal. This project will help to find out when this change in numerical algorithm must be made by comparing the accuracy and parallel performance of the two time stepping schemes in a model which is otherwise identical. A solution to the problem of the convergence of the lines of constant longitude towards the poles is to use a hexagonal-icosahedral grid of the sphere instead of a latitude-longitude grid. This tessellation is more like a football but it divides the sphere into many more hexagons. Parallel performance and accuracy will be compared on the two grid types using a model which is otherwise identical in order to give a clean comparison of the two grids. All code developments will be freely available and open source under the Gnu public licence (GPL). (2) An independent part of the research will be to create new test cases for simplified models of the atmosphere which mimic the strong activity at very small scales that occur in the real world which cannot be resolved in a global model. For example thunderstorms and very steep mountains cannot be fully resolved but can dramatically influence the larger scales. In these new test cases, terms will be added to the equations which mimic these influences at just a single point. These forcings will generate grid scale oscillations or other unrealistic behaviour in inadequate models which otherwise might appear to work well for other existing simple test cases. This could prevent inadequate models from being adopted by operational centres.