Method for improving forecast statistics in large-scale variational data assimilation problems

Over the past two decades, methods of data assimilation have become vital tools for analysis and prediction of complex phenomena in the atmosphere and the ocean. Probably, the most well known application of these methods is the short- and medium-range weather prediction which has become a routine service guiding our everyday life. A wider outlook reveals the great economic and security value of such forecasts. The latest examples may include the volcano ash cloud dispersion in the atmosphere and oil spill tracking in the ocean. The importance of precise monitoring and prediction of such unfortunate events hardly requires any further promotion. The data assimilation methods are, essentially,
mathematical methods which allow the combination of mathematical models of the atmospheric and oceanic flows, observation data (measured by weather stations or by satellites) and some additional information which reflects our experience with these phenomena, accumulated during decades of observations and analysis. A distinguishing feature of the data assimilation problems in meteorology and oceanography is their large-scale, which means that flow models contain a huge number of variables, say $\sim 10^9$ and more. That is why, despite a very significant increase in computational power, the researchers and practitioners still have a very limited choice of suitable mathematical and, subsequently,
algorithmic tools for solving these problems. Among few methods feasible for solving these problems the variational data assimilation method called '4D-Var' is the preferred method implemented at some major operational centers, such as the UK Met Office, ECMWF, Meteo France, etc. While the forecasts produced by this method are usually reasonably good, the problem of evaluating the forecast covariance matrix in the 4D-Var framework currently remains unsolved. This covariance matrix is needed to assess how good and usable the forecast is; without such assessment the forecast itself may be of little
value. The technique currently adopted may not provide the forecast covariance matrix of a reasonable quality when the observed phenomena are highly nonlinear (which is the case in most circumstances). Let us stress again that there are methods in mathematics and statistics capable of solving this problem in principle; it is the problem's dimensionality that makes most of them unusable. Therefore, the main objective of this research is to develop a computationally feasible technique for evaluating an accurate approximation of the forecast covariance matrix in the 4D-Var framework. The suggested methodology to achieve this objective will be based on the latest results recently published by the applicant in high impact computational journals. It represents a fine balance between deterministic and statistical (Bayesian) methods used for solving data assimilation problems and has a potential for the eventual operational use in real-life applications. This potential has been confirmed both by the reviewers of the published papers and in private communications involving some leading experts in the field. However, in its current state the method needs a theoretical and algorithmic upgrade and, importantly, more substantial verification with a realistic large-scale models, such as the NEMO ocean model, which is rapidly becoming popular. All these steps will be conducted in the proposed research.

Beneficiaries from industry will include meteorological and oceanographic centres which produce global operational weather and ocean forecasts, such as Met Office and ECMWF, or regional ocean forecasts such as National Oceanographic Centre (Southampton) and Proudman Oceanographic Laboratory (Liverpool). There are also local air and sea water quality monitoring centres run by City Councils, which rely on global and regional scale forecasts. Other industrial beneficiaries may include DEFRA (groundwater/river hydraulics) or Shell (reservoir modelling), for example. Let us note that the field of environmental modeling is rapidly expanding and one should anticipate that many new relevant application areas will arise in the near future. The results of this research are meant to improve the forecasts and forecast uncertainty estimates, particularly in the situations when the strongly nonlinear phenomena take place (these are often associated with the extreme events). Therefore, all direct and indirect consumers of the weather and ocean forecasts may eventually benefit. Those include as the general public, so
busineses in the UK and abroad, either directly relying on the forecasts (aviation, shipping, fishing), or indirectly (insurance, agriculture).