Learning for Reliable Autonomous Systems and Control

Recent years have seen an increased interest in the use of data, together with machine learning techniques, to develop systems that operate autonomously in the physical world around us. This includes applications ranging from robotic systems or autonomous vehicles, to the industrial Internet-of-Things. Data-driven methods can complement traditional model-based approaches and are particularly suitable for engineering problems where prior physical models, e.g., from first principles, are unavailable, as well as in situations where the environment in which the systems operate is changing over time. However, despite the need for data-driven methodologies alongside model-based ones, the research community is still trying to understand how to employ such methods safely and reliably. When relying on limited amount of data, which is often the case in scenarios where data is collected from the physical world, the outputs of the machine learning procedure will have a mismatch and not perform equally well compared to the case where a model is completely known. Furthermore, recent evidence shows that the outputs of the machine learning procedure can behave unexpectedly when faced with perturbations which can be random or even malicious.

This project aims to fundamentally understand the issues of robustness and reliability in order to enable the development of autonomous systems that can be trusted. To answer these questions, the project offers a new perspective by combining methods from robust control theory and robust optimisation with data-driven methods. For the latter, of particular interest are reinforcement learning methods, because of the need to design autonomous systems that operate in dynamic environments and learn from collected data trajectories over time. Robust control and robust optimization techniques have been traditionally developed for the study of models of dynamical systems with uncertainty, while in contrast this project will explore their use when uncertainty is stemming from the use of data and machine learning techniques. As a result, the objective of the project is twofold. First, it will provide a mathematical framework for characterising fundamentally robustness in reinforcement learning approaches and understanding what the worst-case impact of potential perturbations is. Second, it will develop a methodology for counteracting lack of robustness by rethinking the way data are used to design dynamic autonomous systems. This will be achieved with new algorithms that take into account the developed notions of robustness during the learning phase. These algorithms will be primarily developed and tested on a computer simulation environment for multi-robot systems operating in unknown and dynamic environments, with the potential for experimental validations at a later stage on a proof-of-concept platform.

This project falls within the EPSRC Artificial intelligence and robotics theme, and additionally makes strong connections with the EPSRC Engineering research area (Control Engineering).