Optimisation-centric Generalisations of Bayesian Inference

Large scale black box statistical models are ubiquitous in modern society; and aimed at providing a way to examine the behaviour of complex systems. For example, Improbable has helped design such models as part of the RAMP initiative to help the UK government predict the spread of the COVID-19 virus. In engineering, so-called 'digital twins' of real-world physical phenomena or assets are commonly used to conduct virtual stress tests and predict the behaviour of critical systems in the presence of exogenous shocks.

An important concern for these models is the nature of our uncertainty about their predictions and recommendations. Unlike for more traditional statistical analysis, the underlying models are often highly complex, not immediately interpretable, and often misspecified. As a consequence, standard Bayesian methods of uncertainty quantification derived under the assumptions of the traditional paradigm for statistical analysis are often inappropriate. More specifically, they often result in over-confidence and a lack of robustness.

To tackle this issue, generalised forms of Bayesian uncertainty quantification have recently been developed. Such methods can ensure robustness and reduce the computational burden relative to standard Bayesian methods. This makes them ideal for applications in simulation-based modelling scenarios---such as COVID-19 modelling or digital twins. Yet, to date they have not been used in this context and primarily enjoyed success in time-ordered problems (such as on-line learning, changepoint detection, or filtering and smoothing) as well as in Bayesian Deep Learning applications (such as Bayesian neural networks or deep Gaussian Processes). In spite of their promise however, both their foundational theoretical properties as well as their computation are under-explored topics of research.

In this fellowship, I will advance the theory, methodology, and application of generalised Bayesian posteriors that are defined implicitly through an optimisation problem. While such generalised Bayesian methods have shown great promise, a thorough investigation of this kind will be required if they are to be adopted more widely. As part of this, I will investigate the fundamental question of how one should choose between different generalised posteriors. Complementing this, I will devise methodology for Bayesian computation geared towards the special properties of these posteriors. I will then leverage the advances made as part of this research to apply them on two classes of high-impact problems that traditional Bayesian methods struggle with: models revolving around intractable likelihoods, and simulator-based inference. For the applied component of this research programme, I will draw on the expertise of my project partners and use generalised posteriors for better uncertainty quantification in 'digital twins', as well as applications of importance for national security---such as modelling the COVID-19 pandemic.