# Sequential Monte Carlo methods for applications in high dimensions.

Lead Research Organisation:
University College London

Department Name: Statistical Science

### Abstract

Sequential Monte Carlo (SMC) methods are nowadays routinely employed across a wide range of disciplines to calibrate mathematical models and carry out forecasting about complex non-linear stochastic systems, using information from incoming data. SMC methods have been successfully applied in such diverse areas as econometrics, communications, target tracking, computer vision, roboting and biology. They will infer about unknown parameters of stochastic systems and unobserved states of the systems (the "signal") given streams of data.

However, it is a common knowledge that standard SMC methods cannot tackle important high-dimensional problems, arising in fields such as atmospheric sciences, oceanography, hydrology and signal processing, as their computational cost has been found to increase exponentially fast with the dimension of the state space of the system.

The proposed research will investigate and develop advanced SMC methods of improved algorithmic efficiency in high dimensions, rendering SMC methodology practically relevant in such contexts. This is of high importance as alternative methods currently used in high dimensions cannot fully capture non-linear model dynamics arising in applications, and can give inaccurate estimates of uncertainty or forecasts in such non-linear scenarios.

However, it is a common knowledge that standard SMC methods cannot tackle important high-dimensional problems, arising in fields such as atmospheric sciences, oceanography, hydrology and signal processing, as their computational cost has been found to increase exponentially fast with the dimension of the state space of the system.

The proposed research will investigate and develop advanced SMC methods of improved algorithmic efficiency in high dimensions, rendering SMC methodology practically relevant in such contexts. This is of high importance as alternative methods currently used in high dimensions cannot fully capture non-linear model dynamics arising in applications, and can give inaccurate estimates of uncertainty or forecasts in such non-linear scenarios.

### Planned Impact

Filtering and signal-denoising methods are used in a wide range of important applications for the economy and the society ranging from target tracking and missile guidance, to meteorological and oceanographical forecasting. Some indicative main beneficiaries that could be mentioned are the following:

- Meteorology

- Oceanography

A potential development of computationally efficient SMC methods for data assimilation procedures in such areas will be of great significance, since it will allow practitioners to fully calibrate non-linear stochastic models used in these contexts and provide fully justified measures of uncertainty in such non-linear scenarios..

More analytically, current data assimilation methods at the Met Office, the European Centre for Medium-Range Weather Forecasts and oceanographic centres have been combining carefully formulated Partial Differential Equations capturing physical laws with data arriving from satellites, in situ data and data from many other sources to infer about present states of the atmosphere and parts of the ocean and make forecasts about future ones. Recent efforts is such organisations have looked at Kalman-Filter sequential methods that deliver 'ensembles' of possible future states, thus providing important indications about the uncertainly and the level of confidence at the stated predictions. However, Kalman-Filter methods will have to approximate non-linear model dynamics with linear ones with an effect at the accuracy of the stated forecasts and uncertainly estimates. Thus, discussions have been made in the literature on extending the feasibility of SMC methods in such high-dimensional problems to fully accommodate non-linear model dynamics and increase confidence at stated estimates. The proposed project is expected to make important contributions at this direction.

- Meteorology

- Oceanography

A potential development of computationally efficient SMC methods for data assimilation procedures in such areas will be of great significance, since it will allow practitioners to fully calibrate non-linear stochastic models used in these contexts and provide fully justified measures of uncertainty in such non-linear scenarios..

More analytically, current data assimilation methods at the Met Office, the European Centre for Medium-Range Weather Forecasts and oceanographic centres have been combining carefully formulated Partial Differential Equations capturing physical laws with data arriving from satellites, in situ data and data from many other sources to infer about present states of the atmosphere and parts of the ocean and make forecasts about future ones. Recent efforts is such organisations have looked at Kalman-Filter sequential methods that deliver 'ensembles' of possible future states, thus providing important indications about the uncertainly and the level of confidence at the stated predictions. However, Kalman-Filter methods will have to approximate non-linear model dynamics with linear ones with an effect at the accuracy of the stated forecasts and uncertainly estimates. Thus, discussions have been made in the literature on extending the feasibility of SMC methods in such high-dimensional problems to fully accommodate non-linear model dynamics and increase confidence at stated estimates. The proposed project is expected to make important contributions at this direction.

### Publications

Ehrlich E
(2013)

*Gradient Free Parameter Estimation for Hidden Markov Models with Intractable Likelihoods*in Methodology and Computing in Applied Probability
Beskos A
(2014)

*On the stability of sequential Monte Carlo methods in high dimensions*in The Annals of Applied Probability
Jasra A
(2014)

*Approximate Inference for Observation-Driven Time Series Models with Intractable Likelihoods*in ACM Transactions on Modeling and Computer Simulation
Kantas N
(2014)

*Sequential Monte Carlo Methods for High-Dimensional Inverse Problems: A Case Study for the Navier--Stokes Equations*in SIAM/ASA Journal on Uncertainty Quantification
Kantas N
(2015)

*On Particle Methods for Parameter Estimation in State-Space Models*in Statistical Science
Beskos A
(2015)

*Sequential Monte Carlo methods for Bayesian elliptic inverse problems*in Statistics and Computing
Beskos A
(2016)

*Error Bounds and Normalising Constants for Sequential Monte Carlo Samplers in High Dimensions*in Advances in Applied Probability
Beskos A
(2016)

*On the convergence of adaptive sequential Monte Carlo methods*in The Annals of Applied Probability
Beskos A
(2017)

*A stable particle filter for a class of high-dimensional state-space models*in Advances in Applied Probability
Beskos A
(2017)

*Multilevel sequential Monte Carlo samplers*in Stochastic Processes and their ApplicationsDescription | A main objective of the funded project has been the investigation of the potential of a Monte-Carlo method (Sequential Monte-Carlo) in the context of high-dimensional inverse problems, with important applications in weather forecasting and data assimilation. We have indeed now developed such Monte-Carlo methods which can be appropriate for the problems at hand, and have shown analytical results from their application together with important theoretical findings in 7 published papers. We believe that our research on this problem could be very influential in the area of Monte-Carlo methods as it has gone against the common belief that such methods are not relevant for high-dimensional applications. The research during the Grant has opened up a number of interesting research routes which are now investigated in a number of projects with collaborators. |

Exploitation Route | The research during the Grant had done (at the time) the first steps for demonstrating the potential of principled ensemble Monte-Carlo methods in applications in high dimensions. Further developments since the time of the Grant has shifted attention of many researchers in the Monte-Carlo community in problems in high-dimensions, and the consensus nowadays is that Sequential Monte-Carlo methods can tackle models of much higher dimension that previously thought. |

Sectors | Digital/Communication/Information Technologies (including Software),Environment,Financial Services, and Management Consultancy |

Description | Collaboration with Ajay Jasra at National University of Singapore |

Organisation | National University of Singapore |

Department | Department of Statistics and Applied Probability |

Country | Singapore |

Sector | Academic/University |

PI Contribution | Co-authoring of research output. |

Collaborator Contribution | Co-authoring of research output. |

Impact | A number of publications have resulted through this collaboration. |

Start Year | 2013 |

Description | Collaboration with Dr Kody Law at Oak Ridge National Laboratory (USA) |

Organisation | Oak Ridge National Laboratory |

Country | United States |

Sector | Public |

PI Contribution | Development of research and co-authorship of papers. |

Collaborator Contribution | Development of research and co-authorship of papers. |

Impact | A number of publications. |

Start Year | 2014 |