Autonomous Behaviour and Learning in an Uncertain World

Lead Research Organisation: University of Oxford
Department Name: Engineering Science


The key challenges facing research, development and deployment of autonomous systems require principled solutions in order for scalable systems to become viable. This proposal intertwines probabilistic (Bayesian) inference, model-predictive control, distributed information networks, human-in-the-loop and multi-agent systems to an unprecedented degree. The project focuses on the principled handling of uncertainty for distributed modelling in complex environments which are highly dynamic, communication poor, observation costly and time-sensitive. We aim to develop robust, stable, computationally practical and principled approaches which naturally accommodate these real-world challenges.

Our proposed framework will enable significant progress to be made in a large number of areas essential to intelligent autonomous systems, including 1) the assessment of reliability and fusion of disparate sources of data, 2) allow active data selection based on Bayesian sequential decision making under realistic time, information & computation constraints, 3) allow the advancement of Bayesian reinforcement algorithms in complex systems, and 4) extend Model predictive control (MPC) to probabilistic settings using Gaussian process non-parametric models.

At the systems level, these developments will permit the design of overarching methods for 1) controlled autonomous systems which interact and collaborate, 2) integration of sensing, inference, decision making and learning in acting systems and 3) design methods for validation and verification of systems to enhance robustness and safety.

The ability to meet these objectives depends on a multitude of recent technical developments. These include, 1) development of practical non-parametric algorithms for on-line learning and adaptation 2) approximate inference for Bayesian sequential decision making under constraints, 3) the development of sparse data selection and sparse representation methods for practical handling of large data sets with complex decentralised systems and 4) the implementation of and deployment on powerful modern parallel architectures such as GPUs.

We aim to build on our expertise in Bayesian machine learning, multi-agent systems and control theory and by drawing together closely related developments in these complementary fields we will be able to make substantial improvements to the way artificial agents are able to learn and act, combine and select data sources intelligently, and integrate in robust ways into complex environments with multiple agents and humans in the loop.


10 25 50

publication icon
Anghel M (2013) Algorithmic Construction of Lyapunov Functions for Power System Stability Analysis in IEEE Transactions on Circuits and Systems I: Regular Papers

publication icon
Calliess, J. (2014) Conservative collision prediction and avoidance for stochastic trajectories in continuous time and state. in International Conference on Autonomous Agents and Multiagent Systems (AAMAS2014)

publication icon
Calliess, J. (2013) Nonlinear adaptive hybrid control by combining Gaussian process system identification with classical control laws. in WS- Novel Methods for Learning and Optimization of Control Policies and Trajectories for Robotics, ICRA, 2013.

publication icon
Hancock E (2013) Generalised absolute stability and sum of squares in Automatica

publication icon
Kim Samo, Y. L (2016) String and Membrane Gaussian Process in String and Membrane Gaussian Process

publication icon
Kom Samo, Y L (2015) Scalable Nonparametric Bayesian Inference on Point Processes with Gaussian Processes in Scalable Nonparametric Bayesian Inference on Point Processes with Gaussian Processes

publication icon
Lloyd C (2015) Variational Inference for Gaussian Process Modulated Poisson Process in Variational Inference for Gaussian Process Modulated Poisson Process

publication icon
Lloyd, C (2016) Latent Point Process Allocation in Latent Point Process Allocation

publication icon
Osborne, M. (2013) Active learning of model evidence using Bayesian quadrature in Advances in Neural Information Processing Systems 26 (NIPS 2012), December 2012, Lake Tahoe, USA.

publication icon
Papachristodoulou A (2015) Advances in computational Lyapunov analysis using sum-of-squares programming in Discrete and Continuous Dynamical Systems - Series B

publication icon
Peet M (2012) A Converse Sum of Squares Lyapunov Result With a Degree Bound in IEEE Transactions on Automatic Control

publication icon
Raman DV (2017) Delineating parameter unidentifiabilities in complex models. in Physical review. E

publication icon
Valmorbida G (2017) Nonlinear Static State Feedback for Saturated Linear Plants via a Polynomial Approach in IEEE Transactions on Automatic Control

publication icon
Villaverde AF (2016) Structural Identifiability of Dynamic Systems Biology Models. in PLoS computational biology

publication icon
Y_L. Kom Samo (2015) Generalized Spectral Kernels

publication icon
Zhang X (2015) Improving the Performance of Network Congestion Control Algorithms in IEEE Transactions on Automatic Control

Description 1) Sparse efficient sampling based on informatics criteria can provide stable control algorithms and enable scalable multi-agent coordination.
2) Control mechanisms may be learned from data without a physical mechanism known
3) Guarantees of stability may be derived for probabilistic control methods
4) Bayesian optimisation allows for rapid learning of unknown functions.
Exploitation Route via existing industry partners industry partners & academic publication.
Sectors Aerospace, Defence and Marine,Digital/Communication/Information Technologies (including Software),Energy,Other

Description Our control models have been integrated into drilling simulation by the industrial partner Schlumberger. These models show how, using sparse data, we can effectively use AI techniques to improve reliable control model creation - offering formal some guarantees as well. The extensions of this work can be useful in many areas, from finance to autonomous vehicles.
First Year Of Impact 2014
Sector Energy,Financial Services, and Management Consultancy
Impact Types Economic

Title active sampling for control systems 
Description sparse observations for active control 
Type Of Material Computer model/algorithm 
Year Produced 2013 
Provided To Others? No  
Description Collaboration with Schlumberger 
Organisation Schlumberger Limited
Department Schlumberger Cambridge Research
Country United Kingdom 
Sector Academic/University 
PI Contribution working closely to ensure industrial relevance and disseminate materials
Collaborator Contribution providing data and expertise - funding two studentships as a knock on from this project
Impact papers, software and know-how
Start Year 2012