Efficient Inference for Higher-Order Probabilistic Programs

Lead Research Organisation: University of Oxford
Department Name: Autonom Intelligent Machines & Syst CDT


Context of the research proposal:

Many scientific models can be naturally expressed as stochastic simulators. Probabilistic programming allows users to exploit the source code information of these simulators to conduct Bayesian inference. Full-scale Bayesian inference in general stochastic simulators essentially provides users with a principled way to invert simulators based on observed data. For example, given a simulator that models disease outbreaks and some observed data we can infer the underlying latent parameters which best describe the given disease outbreak.

However, most inference algorithms in Bayesian statistics are designed for models which have a fixed dimensionality. In contrast, higher-order probabilistic programming allows the user to define models which have a variable (possibly even infinite) number of latent variables. The generality of models expressed in higher-order probabilistic programs requires the design of new inference algorithms which are sufficiently general and can exploit the program structure of the simulator. The potential impact of efficient and general Bayesian inference in these simulators would be enormous as it would allow for entirely new scientific workflows of building accurate simulators which can be inverted and improved based on observed data.

Aims and objectives:

Develop novel inference algorithms which are more tailored to a specific probabilistic program based on static analysis of source code
Integrate inference algorithms within popular probabilistic programming environments such as Pyro, Turing or PyProb so that they are accessible to a large number of users

Novelty of the research methodology:

Efficient inference algorithms for higher-order probabilistic programs are still an active area of research. Current approaches are mostly based on Importance Sampling, Sequential Monte Carlo or Variational Inference. We hope to improve upon these approaches and/or potentially unlock entirely new types of inference algorithms.

Alignment to EPSRC's strategies and research areas:

Artificial Intelligence technologies
Programming languages and compilers
Statistic and applied probability
Theoretical computer science


10 25 50

Studentship Projects

Project Reference Relationship Related To Start End Student Name
EP/S024050/1 01/10/2019 31/03/2028
2243853 Studentship EP/S024050/1 01/10/2019 30/09/2023 Tim Reichelt