Novel optimization methods for data assimilation
Lead Research Organisation:
University of Reading
Department Name: Mathematics and Statistics
Abstract
Data assimilation is the process of combining observations with a numerical model in order to estimate the state of a system. It is widely used in many areas of environmental modelling. For example, it is used in weather and ocean forecasting by national weather centres around the world to estimate the initial conditions for a forecast. A common method of data assimilation, known as four-dimensional variational assimilation (4D-Var), formulates the assimilation problem in terms of the optimization of a function that measures the distance between the solution and the actual measurements. This function is then minimized using an iterative numerical method. In practice it is not possible to run such a method to convergence, as there is a limited time in which to do the assimilation and produce a forecast (a forecast must complete before the weather event occurs to be useful). Hence it is important that efficient numerical methods are used that provide the best solution in the time available.
Current operational weather and ocean forecasting systems rely heavily on standard numerical optimization methods. In particular, the common method of incremental variational assimilation, used at centres such as the Met Office, is an approximation to a well-known optimization procedure known as the Gauss-Newton iteration method. More recently, one of the supervisors on this project (Cartis) has developed more advanced optimization methods, called quadratic regularization (QR) and adaptive cubic regularization (ARC). These new methods have been shown to be more efficient on a certain class of problems. The aim of this project is to test if these methods can be advantageous in data assimilation algorithms.
The student will implement the standard optimization methods and two new methods in a series of problems of increasing complexity, starting from a one-variable equation and leading up to the shallow-water equations, a simplified atmospheric model. The performance of the methods will be compared computationally and analysed mathematically. This project bridges the research fields of meteorology and mathematics, essential components of the University's Environment Theme. The outcome of this work will be the application of latest mathematical advances in optimization to the data assimilation problem for real applications, which has a potential for a big impact on the efficiency of operational weather and climate forecasting.
This proposal crosses the mathematical research areas of data assimilation techniques and numerical analysis for weather and climate models. Data assimilation is widely used in weather and ocean prediction and is becoming more important for prediction on longer timescales (seasonal to decadal). As numerical models move to ever higher resolution, the data assimilation systems need to solve a bigger optimization problem and so much more efficient methods are needed. In this project the student will take the latest mathematical advances in numerical optimization and adapt them for use in the data assimilation problem. Methods will be demonstrated on small problems but designed with implementation on real problems in mind, providing a clear path for application to operational systems. This has the potential for a big impact on the efficiency of future operational data assimilation schemes. Specific outcomes of the project will include:
1. an evaluation and mathematical analysis of the applicability of the new optimization algorithms for data assimilation;
2. a demonstration of the new algorithms on a model relevant to atmospheric forecasting;
3. a practical proposal for implementing the new algorithms in large systems, including appropriate termination criteria for the iterative algorithms.
Current operational weather and ocean forecasting systems rely heavily on standard numerical optimization methods. In particular, the common method of incremental variational assimilation, used at centres such as the Met Office, is an approximation to a well-known optimization procedure known as the Gauss-Newton iteration method. More recently, one of the supervisors on this project (Cartis) has developed more advanced optimization methods, called quadratic regularization (QR) and adaptive cubic regularization (ARC). These new methods have been shown to be more efficient on a certain class of problems. The aim of this project is to test if these methods can be advantageous in data assimilation algorithms.
The student will implement the standard optimization methods and two new methods in a series of problems of increasing complexity, starting from a one-variable equation and leading up to the shallow-water equations, a simplified atmospheric model. The performance of the methods will be compared computationally and analysed mathematically. This project bridges the research fields of meteorology and mathematics, essential components of the University's Environment Theme. The outcome of this work will be the application of latest mathematical advances in optimization to the data assimilation problem for real applications, which has a potential for a big impact on the efficiency of operational weather and climate forecasting.
This proposal crosses the mathematical research areas of data assimilation techniques and numerical analysis for weather and climate models. Data assimilation is widely used in weather and ocean prediction and is becoming more important for prediction on longer timescales (seasonal to decadal). As numerical models move to ever higher resolution, the data assimilation systems need to solve a bigger optimization problem and so much more efficient methods are needed. In this project the student will take the latest mathematical advances in numerical optimization and adapt them for use in the data assimilation problem. Methods will be demonstrated on small problems but designed with implementation on real problems in mind, providing a clear path for application to operational systems. This has the potential for a big impact on the efficiency of future operational data assimilation schemes. Specific outcomes of the project will include:
1. an evaluation and mathematical analysis of the applicability of the new optimization algorithms for data assimilation;
2. a demonstration of the new algorithms on a model relevant to atmospheric forecasting;
3. a practical proposal for implementing the new algorithms in large systems, including appropriate termination criteria for the iterative algorithms.
Organisations
Studentship Projects
| Project Reference | Relationship | Related To | Start | End | Student Name |
|---|---|---|---|---|---|
| EP/N509723/1 | 01/10/2016 | 30/09/2021 | |||
| 1792152 | Studentship | EP/N509723/1 | 01/10/2016 | 30/09/2019 | Maha Kaouri |
| Description | I am investigating globally convergent optimisation algorithms which are guaranteed to find a solution to the data assimilation problem. This problem is solved in practice to obtain a numerical weather forecast, and so ensuring the optimisation method used is able to obtain an accurate solution in the time and computational cost available is of utmost importance. The current method used in practice (Gauss-Newton) is known to diverge in some cases. In my research, I have focused on two alternative algorithms, Gauss-Newton with linesearch and Gauss-Newton with quadratic regularisation which are guaranteed to converge, but may come at a cost. In my work, I have studied the convergence properties of these methods, both theoretically (convergence proofs) as well as numerically. I have found that there are cases when the globally convergent methods are able to obtain a solution in a reasonable amount of computational effort when Gauss-Newton cannot. |
| Exploitation Route | The fact that I am showing that alternative methods may be affordable enough to use in practice could be of interest to operational centres. |
| Sectors | Aerospace, Defence and Marine,Agriculture, Food and Drink,Environment |