Autonomous behaviour and learning in an uncertain world
Lead Research Organisation:
University of Cambridge
Department Name: Engineering
Abstract
The key challenges facing research, development and deployment of autonomous systems require principled solutions in order for scalable systems to become viable. This proposal intertwines probabilistic (Bayesian) inference, model-predictive control, distributed information networks, human-in-the-loop and multi-agent systems to an unprecedented degree. The project focuses on the principled handling of uncertainty for distributed modelling in complex environments which are highly dynamic, communication poor, observation costly and time-sensitive. We aim to develop robust, stable, computationally practical and principled approaches which naturally accommodate these real-world challenges.
Our proposed framework will enable significant progress to be made in a large number of areas essential to intelligent autonomous systems, including 1) the assessment of reliability and fusion of disparate sources of data, 2) allow active data selection based on Bayesian sequential decision making under realistic time, information & computation constraints, 3) allow the advancement of Bayesian reinforcement algorithms in complex systems, and 4) extend Model predictive control (MPC) to probabilistic settings using Gaussian process non-parametric models.
At the systems level, these developments will permit the design of overarching methods for 1) controlled autonomous systems which interact and collaborate, 2) integration of sensing, inference, decision making and learning in acting systems and 3) design methods for validation and verification of systems to enhance robustness and safety.
The ability to meet these objectives depends on a multitude of recent technical developments. These include, 1) development of practical non-parametric algorithms for on-line learning and adaptation 2) approximate inference for Bayesian sequential decision making under constraints, 3) the development of sparse data selection and sparse representation methods for practical handling of large data sets with complex decentralised systems and 4) the implementation of and deployment on powerful modern parallel architectures such as GPUs.
We aim to build on our expertise in Bayesian machine learning, multi-agent systems and control theory and by drawing together closely related developments in these complementary fields we will be able to make substantial improvements to the way artificial agents are able to learn and act, combine and select data sources intelligently, and integrate in robust ways into complex environments with multiple agents and humans in the loop.
Our proposed framework will enable significant progress to be made in a large number of areas essential to intelligent autonomous systems, including 1) the assessment of reliability and fusion of disparate sources of data, 2) allow active data selection based on Bayesian sequential decision making under realistic time, information & computation constraints, 3) allow the advancement of Bayesian reinforcement algorithms in complex systems, and 4) extend Model predictive control (MPC) to probabilistic settings using Gaussian process non-parametric models.
At the systems level, these developments will permit the design of overarching methods for 1) controlled autonomous systems which interact and collaborate, 2) integration of sensing, inference, decision making and learning in acting systems and 3) design methods for validation and verification of systems to enhance robustness and safety.
The ability to meet these objectives depends on a multitude of recent technical developments. These include, 1) development of practical non-parametric algorithms for on-line learning and adaptation 2) approximate inference for Bayesian sequential decision making under constraints, 3) the development of sparse data selection and sparse representation methods for practical handling of large data sets with complex decentralised systems and 4) the implementation of and deployment on powerful modern parallel architectures such as GPUs.
We aim to build on our expertise in Bayesian machine learning, multi-agent systems and control theory and by drawing together closely related developments in these complementary fields we will be able to make substantial improvements to the way artificial agents are able to learn and act, combine and select data sources intelligently, and integrate in robust ways into complex environments with multiple agents and humans in the loop.
Planned Impact
Autonomous intelligent systems are expected to have an increasing effect on our lives. Initially they will be used within highly technical systems, such as robots for handling radioactive materials in power stations, for performing military reconnaissance without risking soldiers' lives, for helping surgeons, or for excavating building sites. Later they are likely to become used in situations which involve interaction with the general public, for example to provide assistance to the elderly or disabled, or perhaps to take over the driving of cars.
In order to allow any of these things to be done successfully and safely, research is needed in some underpinning engineering and science. The biggest challenge is that of dealing with the huge amount of uncertainty that will be faced by autonomous intelligent systems - uncertainty that humans can often deal with pretty well, but that we can still handle to only a limited degree in our mathematics and engineering. Therefore the scientific core of our project rests on the use of probability theory and in particular on the use of Bayes' theorem, and a whole family of methods that result from it, for calculating and keeping track of uncertainty in very complex situations. This is necessary if autonomous agents are to act intelligently in non-trivial situations.
Many people have already done plenty of work on 'Bayesian methods', so what is new in this project? The novelty is the use of these methods as a 'glue' to tie together the many and various technologies that are needed to make autonomous intelligent systems work. Some of these (such as flying an aeroplane) are well known and can achieve great precision; others (such as distinguishing a pedestrian in the road from a shadow, or an underground cable from a rock) are still at a relatively early stage of evolution. A key capability required of autonomous intelligent systems is that of planning, and re-planning in the face of incoming evidence and measurements, how to perform or complete a task. This requires some understanding of behaviours, both of one's own system (eg 'how does this aeroplane work?') and of one's environment (eg 'where will that storm go next?'). The methods currently used to represent and analyse these different aspects differ a lot in their mathematical forms. Our ambition is to develop methods, with a Bayesian basis, which will allow them all to be used with each other.
Our project will investigate in detail how to integrate such technologies in a way that enables them all to work together. In addition to discovering how to do this, we will also train a number of researchers in the required techniques, spread the word to other academic researchers, and share our results with a number of companies who are trying to develop autonomous intelligent systems.
In order to allow any of these things to be done successfully and safely, research is needed in some underpinning engineering and science. The biggest challenge is that of dealing with the huge amount of uncertainty that will be faced by autonomous intelligent systems - uncertainty that humans can often deal with pretty well, but that we can still handle to only a limited degree in our mathematics and engineering. Therefore the scientific core of our project rests on the use of probability theory and in particular on the use of Bayes' theorem, and a whole family of methods that result from it, for calculating and keeping track of uncertainty in very complex situations. This is necessary if autonomous agents are to act intelligently in non-trivial situations.
Many people have already done plenty of work on 'Bayesian methods', so what is new in this project? The novelty is the use of these methods as a 'glue' to tie together the many and various technologies that are needed to make autonomous intelligent systems work. Some of these (such as flying an aeroplane) are well known and can achieve great precision; others (such as distinguishing a pedestrian in the road from a shadow, or an underground cable from a rock) are still at a relatively early stage of evolution. A key capability required of autonomous intelligent systems is that of planning, and re-planning in the face of incoming evidence and measurements, how to perform or complete a task. This requires some understanding of behaviours, both of one's own system (eg 'how does this aeroplane work?') and of one's environment (eg 'where will that storm go next?'). The methods currently used to represent and analyse these different aspects differ a lot in their mathematical forms. Our ambition is to develop methods, with a Bayesian basis, which will allow them all to be used with each other.
Our project will investigate in detail how to integrate such technologies in a way that enables them all to work together. In addition to discovering how to do this, we will also train a number of researchers in the required techniques, spread the word to other academic researchers, and share our results with a number of companies who are trying to develop autonomous intelligent systems.
Publications
Bauer Matthias
(2016)
Understanding Probabilistic Sparse Gaussian Process Approximations
in arXiv e-prints
Bischoff B
(2014)
Policy search for learning robot control using sparse data
Calandra R
(2016)
Manifold Gaussian Processes for regression
Calliess J.
(2015)
Lipschitz Constant Estimation and Quadrature
Calliess J.
(2017)
Lipschitz Optimisation for Lipschitz Interpolation
Calliess Jan-Peter
(2016)
Lazily Adapted Constant Kinky Inference for Nonparametric Regression and Model-Reference Adaptive Control
in arXiv e-prints
Description | We have developed new machine learning algorithms which can learning from noisy data. Special focus has been on approximate algorithms which can handle very large amounts of data and on Reinforcement Learning algorithms which can learn to act in uncertain, and unknown environments. |
Exploitation Route | Better understanding of machine learning algorithms and better, more practical implementations of machine learning algorithms are an urgent requirement for widespread use of these modern techniques in science and engineering. |
Sectors | Digital/Communication/Information Technologies (including Software) Other |
Description | The application of modern machine learning algorithms in real world industrial settings promises significant gains in efficiency through automation, speed and accuracy. A key challenge in the widespread application of autonomous systems is representation and reasoning in the presence of noise and uncertainty. In this project, we tackled a number of challenging problems including modelling, adaptation, prediction, control and decision making in data rich environments using limited computational resources. The main outcomes are focused primarily on two key areas: the development and software implementation of new efficient, automatic and robust algorithms and the understanding of the applicability of flexible data driven methods in challenging industrial environments. In the first area, we have further developed the state of the art in probabilistic modelling for noisy and uncertain environments. We have been especially focused on automation of the modelling process, as this is a key requirement to routine deployment where reliability is paramount. These outcomes have been published. The second objective has been testing our algorithms with our industrial partner Schlumberger Research. This data domain is extremely challenging, characterised by very limited instrumentation, limited communication, noisy and fault prone data and quick response requirements. The developed methodologies were testing on both historical recordings and to data gathered live at a test site, for the purposes of exploring the system performance in real time. The outcomes have shown that these modern, data driven probabilistic approaches are viable and able to make useful diagnosis and predictions, but that implementing and running these methods in challenging circumstances is not trivial. Especially the integration with existing methodologies is highlighted, both in terms of integrating first principles physical modeling with data driven approaches and in terms of practically integrating modelling, prediction and decision making into the current process control loops. The project has shown what is possible using and further developing state of the art methodologies in a challenging industrial sector. Valuable experience has been gained on both the academic and the industrial side of the partmentship. |
First Year Of Impact | 2015 |
Sector | Digital/Communication/Information Technologies (including Software),Education,Energy |
Impact Types | Economic |
Description | Arno Solin |
Organisation | Aalto University |
Country | Finland |
Sector | Academic/University |
PI Contribution | Research collaboration, combining our expertise, mainly focusing on indoor localisation using magnetometers. |
Collaborator Contribution | Research collaboration, combining our expertise, mainly focusing on indoor localisation using magnetometers. |
Impact | Submission of two papers: 1) Arno Solin, Manon Kok, Niklas Wahlström, Thomas B. Schön and Simo Särkkä, Modeling and interpolation of the ambient magnetic field by Gaussian processes. Conditionally accepted to IEEE Transactions on Robotics, 2017. 2) Manon Kok and Arno Solin, Rank-reduced Gaussian process maps for magnetic field SLAM, 2018. |
Start Year | 2015 |
Description | Schlumberger |
Organisation | Schlumberger Limited |
Department | Schlumberger Cambridge Research |
Country | United Kingdom |
Sector | Academic/University |
PI Contribution | Research into the use of probabilistic methods in model predictive control and machine learning to help understand and control aspects of drilling. Autonomous control is critical as communication in deep wells is very poor. Modelling aspects include the movement of the drill string and the management of well pressure. |
Collaborator Contribution | Sharing of tools and data recorded from actual wells, test wells and simulation. |
Impact | This is an ongoing feasibility study in the application of sophisticated probabilistic methods from model predictive control and machine learning. |
Start Year | 2012 |
Title | Gaussian Process for Machine Learning (GPML) Toolbox, version 4.2 |
Description | Gaussian Processes (GPs) can conveniently be used for Bayesian supervised learning, such as regression and classification. In its simplest form, GP inference can be implemented in a few lines of code. However, in practice, things typically get a little more complicated: you might want to use complicated covariance functions and mean functions, learn good values for hyperparameters, use non-Gaussian likelihood functions (rendering exact inference intractable), use approximate inference algorithms, or combinations of many or all of the above. This is what the GPML software package does. |
Type Of Technology | Software |
Year Produced | 2010 |
Open Source License? | Yes |
Impact | GPML is a free toolbox used widely across the education and industrial sectors. |
URL | http://www.gaussianprocess.org/gpml/code |
Description | ERNSI presentation |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Other academic audiences (collaborators, peers etc.) |
Results and Impact | ERNSI is the European Research Network on System Identification. This was a presentation for the network that provided an overview of our approach to Bayesian optimization and engendered discussion of these methods as well as input from the wide system identification community (in Europe). There was a good deal of discussion as well as input on applications from the system identification community. |
Year(s) Of Engagement Activity | 2014 |
Description | ERNSI workhshop |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Postgraduate students |
Results and Impact | Two poster presentations at the yearly ERNSI workshop on System Identification |
Year(s) Of Engagement Activity | 2017 |
URL | https://ernsi2017.sciencesconf.org/ |
Description | Guest lectures at TU Berlin |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Undergraduate students |
Results and Impact | Two guest lectures in the master course Inertial Sensor Fusion at the TU Berlin |
Year(s) Of Engagement Activity | 2018 |
URL | http://www.control.tu-berlin.de/Teaching:Inertial_Sensor_Fusion |
Description | Invited talk at the University of Sheffield |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Postgraduate students |
Results and Impact | Talk at the University of Sheffield, invited by Prof. Lyudmila Mihaylova |
Year(s) Of Engagement Activity | 2017 |
Description | RSS Invited talk |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Other academic audiences (collaborators, peers etc.) |
Results and Impact | This was a presentation of my work on Bayesian optimization to the Royal Statistical Society's annual meeting. The talk encouraged a great deal of conversation and discussion. The talk introduced our particular approaches and gave a preliminary view of software implementing our techniques. |
Year(s) Of Engagement Activity | 2014 |