Scalable Design of Robust Neural Network Controllers
Lead Research Organisation:
UNIVERSITY OF OXFORD
Abstract
Research description:
Research into neural network models is a large field and requires a variety of mathematical techniques to address any relevant research questions. There has been a recent resurgence of interest due to the increase in the prevalence of big-data and computational power available. Examples of such areas include image recognition, weather prediction and natural language processing. One important consideration is the increasing use of neural networks in safety-critical applications, such as autonomous vehicle technology. This accentuates the biggest shortcoming of neural networks, which is their sensitivity to adversarial inputs: small changes in the input set can lead to large changes in the output. Despite considerable effort from the research community to improve our understanding and to allow certification of neural networks, to date guarantees on these systems are not sufficient for their widespread use in safety-critical applications.
This doctoral project will build upon the existing research to explore various problems related to the robustness of neural networks. One popular method that has seen a large amount of success is to use bounds on the activation functions within these networks to provide such guarantees. However, due to the large number of possible ways to bound the activation functions, there is a trade-off between conservativeness and complexity. It is possible to improve the scalability of optimization problem by using theory from chordal graphs, where large constraints matrices are split into equivalent smaller constraints matrices. These ideas can also be combined with Sum of Squares programming - a technique that uses semi-definite programming. This technique can be used to obtain tighter bounds on the neural network output, whilst maintaining a computational scalable method of obtaining a solution. These ideas can also be extended to neural network controllers, to provide better control performance and robustness of a feedback system.
Aims and objectives:
The end goal of this work is to create a framework to design robust neural network controllers in a scalable way. The robustness can be quantified using stability theory and determined through solving a Sum of Squares program. Since neural network structures can become very large, determining the stability can be computational expensive. However, there are ways to reformulate the problem to reduce this computational burden by using ideas from chordal sparsity. These techniques are a key area that is being explored in this project. The main questions that will be addressed in this doctoral project focus on combining Sum of Squares programming and chordal sparsity to the neural network verification problem. Once this is established the next objective is to see how these ideas extend to neural network controllers and then how they can be used to improve the performance of feedback systems.
Novelty of the research methodology:
Sum of Squares techniques have not yet been applied to problems surrounding neural networks. This has led to gaps in the research area, which will be explored in this DPhil. Combining Sum of Squares and sparsity exploiting methods is an open research area. Neural network controllers are also emerging in the research community and there are many questions that need exploring.
Research into neural network models is a large field and requires a variety of mathematical techniques to address any relevant research questions. There has been a recent resurgence of interest due to the increase in the prevalence of big-data and computational power available. Examples of such areas include image recognition, weather prediction and natural language processing. One important consideration is the increasing use of neural networks in safety-critical applications, such as autonomous vehicle technology. This accentuates the biggest shortcoming of neural networks, which is their sensitivity to adversarial inputs: small changes in the input set can lead to large changes in the output. Despite considerable effort from the research community to improve our understanding and to allow certification of neural networks, to date guarantees on these systems are not sufficient for their widespread use in safety-critical applications.
This doctoral project will build upon the existing research to explore various problems related to the robustness of neural networks. One popular method that has seen a large amount of success is to use bounds on the activation functions within these networks to provide such guarantees. However, due to the large number of possible ways to bound the activation functions, there is a trade-off between conservativeness and complexity. It is possible to improve the scalability of optimization problem by using theory from chordal graphs, where large constraints matrices are split into equivalent smaller constraints matrices. These ideas can also be combined with Sum of Squares programming - a technique that uses semi-definite programming. This technique can be used to obtain tighter bounds on the neural network output, whilst maintaining a computational scalable method of obtaining a solution. These ideas can also be extended to neural network controllers, to provide better control performance and robustness of a feedback system.
Aims and objectives:
The end goal of this work is to create a framework to design robust neural network controllers in a scalable way. The robustness can be quantified using stability theory and determined through solving a Sum of Squares program. Since neural network structures can become very large, determining the stability can be computational expensive. However, there are ways to reformulate the problem to reduce this computational burden by using ideas from chordal sparsity. These techniques are a key area that is being explored in this project. The main questions that will be addressed in this doctoral project focus on combining Sum of Squares programming and chordal sparsity to the neural network verification problem. Once this is established the next objective is to see how these ideas extend to neural network controllers and then how they can be used to improve the performance of feedback systems.
Novelty of the research methodology:
Sum of Squares techniques have not yet been applied to problems surrounding neural networks. This has led to gaps in the research area, which will be explored in this DPhil. Combining Sum of Squares and sparsity exploiting methods is an open research area. Neural network controllers are also emerging in the research community and there are many questions that need exploring.
Planned Impact
The UK is faced with an increasing skills shortage, with a recent (2012) large-scale survey reporting that half of all key UK industries surveyed suffer from a worsening skills shortage. This is even more acute in high-tech industry and requires core investment in teaching highly-qualified cohorts, not only the foundational theoretical underpinning in this CDT's remit, but also the acumen to bring this theory to bear on a range of real problems. This CDT will promote training in transformative research that will revolutionise and intertwine theory and practice. If we are to train a generation of researchers to lead in the use of pervasive computation we must actively promote interconnecting research areas. The CDT directly addresses the Autonomous Systems & Robotics priority area and interlinks with priorities in Digitally Connected Citizens, New Digital Ventures, and smart Energy Systems and Digital Healthcare. Furthermore, the CDT has strong links to several current EPSRC challenge themes: 1) Manufacturing the Future: Sustainable manufacturing can only be achieved via autonomy, and machine intelligence at global scale. In today's market, the UK's competitive advantage lies in training highly-skilled researchers that will be able to pioneer distributed autonomous systems into manufacturing processes. 2) Energy: Intelligence and autonomy are key to energy-efficient driving and transportation systems, smart energy grids and efficient use of sparse resources. 3) Digital Economy: Intelligent machines and systems can assist people and give them control over their lives in a number of contexts, such as assisted living, home healthcare, transportation, skill & knowledge transfer and telepresence. 4) Living with Environmental Change: Intelligent hand-held devices and participatory sensing will extend environmental monitoring to unprecedented spatial and temporal scales, building real sensor systems and citizen science platforms to monitor the environment, pollutants and biodiversity.
The CDT will allow us to bring together our collaborations with industrial partners into a unique consortium, which will underpin the student training program, from fundamentals to development, deployment and use. The CDT has secured support not only from the University, but also from a team of industrial partners, who share our vision. We have support from an impressive list of companies, from global multi-nationals and large corporations, such as BAE Systems, BP, Schlumberger & YouGov (internships, studentships and membership of the external steering group), Microsoft, Google, Honeywell, Ascending Technologies, SciSys & Man Group (internships & part of our external steering group), ABB, Infosys, QinetiQ (internships and studentships). Industry and commerce will have an active participation in the CDT programme via internships and studentships; provision of short lectures highlighting the practical application of the taught material; proposing first-year research projects; membership of the steering committee; industrial placements into Oxford. Industrial participation, at all levels, will enhance the quality of the training programme and provide access to a unique pool of CDT talent. We believe that our approach to industrial engagement places realistic requirements on both industry and students.
The benefits of the CDT will be many-fold. The students will benefit via a strong foundation in the principles & practice of autonomous & intelligent systems and subsequent research with world-leading groups. The enthusiasm shown by a range of industries indicates an appetite for engaging with the student cohort, promoting clear dissemination, impact and collaboration routes benefiting industry, academia and the UK economy.
The CDT will allow us to bring together our collaborations with industrial partners into a unique consortium, which will underpin the student training program, from fundamentals to development, deployment and use. The CDT has secured support not only from the University, but also from a team of industrial partners, who share our vision. We have support from an impressive list of companies, from global multi-nationals and large corporations, such as BAE Systems, BP, Schlumberger & YouGov (internships, studentships and membership of the external steering group), Microsoft, Google, Honeywell, Ascending Technologies, SciSys & Man Group (internships & part of our external steering group), ABB, Infosys, QinetiQ (internships and studentships). Industry and commerce will have an active participation in the CDT programme via internships and studentships; provision of short lectures highlighting the practical application of the taught material; proposing first-year research projects; membership of the steering committee; industrial placements into Oxford. Industrial participation, at all levels, will enhance the quality of the training programme and provide access to a unique pool of CDT talent. We believe that our approach to industrial engagement places realistic requirements on both industry and students.
The benefits of the CDT will be many-fold. The students will benefit via a strong foundation in the principles & practice of autonomous & intelligent systems and subsequent research with world-leading groups. The enthusiasm shown by a range of industries indicates an appetite for engaging with the student cohort, promoting clear dissemination, impact and collaboration routes benefiting industry, academia and the UK economy.
Organisations
Publications
Newton M
(2021)
Neural Network Verification using Polynomial Optimisation
Newton M
(2020)
Network Lyapunov Functions for Epidemic Models
Studentship Projects
| Project Reference | Relationship | Related To | Start | End | Student Name |
|---|---|---|---|---|---|
| NE/W502728/1 | 31/03/2021 | 30/03/2022 | |||
| 2077605 | Studentship | NE/W502728/1 | 30/09/2018 | 29/04/2023 | Matthew Newton |
| Description | It has been shown that certain epidemic models change their stability properties when the underlying network of interactions changes. More specifically if the network satisfies a "strong connectivity" property then it will exhibit a certain type of stability, however this changes when this property is lost. It has been shown that by using the structure of neural networks, the computational time taken to prove their robustness can be reduced dramatically. Meaning that neural networks that were too large to be verified as robust can now be verified using this property. Related to this there has been an additional finding that shows a more accurate method of proving the robustness of neural networks can be used. The problem has also been reformulated to simultaneously exploit the sparsity in the problem, whilst improving the accuracy. Neural networks have also seen a recent increased use in control feedback systems, since they have the potential to improve the performance of these systems compared to traditional controllers, due to their ability to act as general function approximators. However, since feedback systems are usually subject to external perturbations and neural networks are sensitivity to small changes, providing robustness guarantees has proven challenging. Non-linear systems that contain neural network controllers have been analysed. Firstly it has been found that the accuracy of outer-approximations of the reachable sets can be improved using sparse polynomial optimisation. Secondly, a Sum of Squares programming framework can compute the stability of these systems. Both of these approaches provide better robustness guarantees over existing methods. |
| Exploitation Route | The outcomes of the work on epidemic models so far have added to the academic literacy on understanding the behaviour of epidemic models. The findings related to neural networks can be used to increase the performance of neural network verification, which has a huge range of applications in the real world and is a rapidly developing research field. |
| Sectors | Digital/Communication/Information Technologies (including Software) |