Statistical Decision Making: Advances in Methods and Techniques

This project falls within the EPSRC's Artificial Intelligence and Robotics research area. The aim is to improve decision-making algorithms and Bayesian machine learning methods, for the task of controlling autonomous agents (either physical or software-based).

In the context of decision making, Bayesian modelling provides a robust way of quantifying one's beliefs about the world. Decision-making agents with accurate beliefs about their surrounding environment, and a well-calibrated notion of uncertainty about the results of their actions, can out-perform agents which do not account for uncertainty in this way. For instance, such agents can more effectively balance the need to explore the effects of different behaviours or capitalise on existing knowledge of how to act, to pursue their objectives. These agents can also be more sensitive to the risk that their actions carry, with important implications for safety of such deployments.

Agents often receive high-dimensional sensory information and collect vast numbers of observations. In these settings, Bayesian modelling techniques may suffer from a lack of computational tractability and are often eschewed for the flexibility and speed of neural networks. One of the initial focuses of this project is to develop Bayesian deep learning methods for use in this setting, which retain the representational ability of deep neural networks, while also retaining accurate predictive uncertainty, and being readily optimisable.

The second focus of this project is to fruitfully use these new Bayesian methods in conjunction with decision-making algorithms, to improve the control of autonomous agents. The practitioner defines some objective measure of the agent's performance on a given task, after which the agent is left to explore its environment in a self-directed way, with the goal of learning how to act to maximise its objective. Important research problems include how to improve the agent's sample efficiency (that is, how effectively the agent can gather experience to maximise its objective), sensitivity to risk, and ability to plan sequences of actions in advance, by accounting for causal structures in the environment.

In recent years, such agents have already reached or even surpassed human-level performance in a range of non-trivial tasks. Leveraging the ability of neural networks to learn effective representations of the environment gives us purchase on an ever more challenging class of control problems, which have hitherto been difficult to model with traditional methods. The scope for responsible and effective automation of real-world problems provides further motivation for the development of these methods.