Resilient Path Coordination in Connected Vehicle Systems
Lead Research Organisation:
University of Cambridge
Department Name: Computer Science and Technology
Abstract
The deployment of connected and autonomous vehicles presents us with transformational opportunities for road transport. As the standardization of inter-vehicular communications progresses, vehicles will soon be wirelessly connected, enabling coordinated driving strategies. By enabling vehicles to jointly agree on optimal maneuvers and navigation strategies, coordinated driving promises to improve overall traffic throughput, road capacity, and passenger safety. However, coordinated driving in connected and autonomous vehicle systems suffers from one key limitation: all vehicles in the system are assumed to be cooperative. This is an issue, since automated vehicle control systems are susceptible to numerous failure conditions, ranging from internally triggered faults (e.g., hardware, software, or communications failures) to externally triggered faults (e.g., environmental disturbances or malicious tampering). When cooperation breaks down due to these faults, coordinated driving strategies lead to negative unanticipated consequences, compromising passenger safety and traffic fluidity.
The goal of this project is to develop a resilient coordinated driving method for multi-vehicle systems that are potentially non-cooperative and unreliable. The issue of providing resilience in the face of non-cooperation (e.g., faulty, byzantine or adversarial agents) has received considerable attention within the domain of distributed network control. However, it is not until very recently that we have started to tackle the question of how to deal with failures and misbehavior when connected agents (vehicles or robots) are mobile. Although preliminary results are promising, the methods deal with control (without planning), and cannot handle discrete workspace constraints (i.e., lane topographies) nor kinodynamic constraints (i.e., car-like motion primitives). Consequently, they do not lend themselves to the problem of coordinated driving. There lies a gap between what we know about resilient network control, and what we know about resilient path coordination. The main contribution of this research programme is to fill this gap by providing methods for resilient planning of trajectories for car-like vehicles with lane constraints.
Our methodology is based on a juxtaposition of centralized and decentralized path planning: we will leverage decentralized planning to guarantee collision-free paths at all times and in all circumstances, and couple this with centralized planning that optimizes global objectives whenever possible. The aim is to develop an adaptive algorithm that slides between the distinct modes as a function of real-time factors that define the level of cooperation in the multi-vehicle system. Such sliding mode architectures have yet to be established within the context of connected multi-vehicle systems, where vehicles cannot be assumed to be cooperative at all times. The proposed research will build upon the expertise of the PI in the field of resilient control, inter-vehicular coordination, and optimization.
The implications of this research are expected to contribute to the theory of multi-vehicle path planning and control, with direct applications to connected and autonomous vehicles. This will ultimately contribute to the improvement of future road transport systems, addressing both safety as well as efficiency.
The goal of this project is to develop a resilient coordinated driving method for multi-vehicle systems that are potentially non-cooperative and unreliable. The issue of providing resilience in the face of non-cooperation (e.g., faulty, byzantine or adversarial agents) has received considerable attention within the domain of distributed network control. However, it is not until very recently that we have started to tackle the question of how to deal with failures and misbehavior when connected agents (vehicles or robots) are mobile. Although preliminary results are promising, the methods deal with control (without planning), and cannot handle discrete workspace constraints (i.e., lane topographies) nor kinodynamic constraints (i.e., car-like motion primitives). Consequently, they do not lend themselves to the problem of coordinated driving. There lies a gap between what we know about resilient network control, and what we know about resilient path coordination. The main contribution of this research programme is to fill this gap by providing methods for resilient planning of trajectories for car-like vehicles with lane constraints.
Our methodology is based on a juxtaposition of centralized and decentralized path planning: we will leverage decentralized planning to guarantee collision-free paths at all times and in all circumstances, and couple this with centralized planning that optimizes global objectives whenever possible. The aim is to develop an adaptive algorithm that slides between the distinct modes as a function of real-time factors that define the level of cooperation in the multi-vehicle system. Such sliding mode architectures have yet to be established within the context of connected multi-vehicle systems, where vehicles cannot be assumed to be cooperative at all times. The proposed research will build upon the expertise of the PI in the field of resilient control, inter-vehicular coordination, and optimization.
The implications of this research are expected to contribute to the theory of multi-vehicle path planning and control, with direct applications to connected and autonomous vehicles. This will ultimately contribute to the improvement of future road transport systems, addressing both safety as well as efficiency.
Planned Impact
The real-world installment of V2V and V2I infrastructure is still in a preliminary stage - we have a window of opportunity to develop reliable and resilient coordinated driving solutions, before large-scale deployment of connected (and autonomous) vehicles takes place.
The benefits of resilient coordinated driving methods are:
(a) gains in road capacity, traffic throughput, and fuel efficiency, without compromising passenger safety; this will be facilitated by our sliding mode path coordination algorithm;
(b) creation and maintenance of user trust in coordinated driving systems, with direct implications on the commercial aspects of involved industries; this will be facilitated by formal guarantees and empirical validations of our methods;
(c) drive the generation of further knowledge; this will be facilitated through our principled study of failure modes in coordinated multi-vehicle systems, and through the publication of benchmark tests.
The benefits of resilient coordinated driving methods are:
(a) gains in road capacity, traffic throughput, and fuel efficiency, without compromising passenger safety; this will be facilitated by our sliding mode path coordination algorithm;
(b) creation and maintenance of user trust in coordinated driving systems, with direct implications on the commercial aspects of involved industries; this will be facilitated by formal guarantees and empirical validations of our methods;
(c) drive the generation of further knowledge; this will be facilitated through our principled study of failure modes in coordinated multi-vehicle systems, and through the publication of benchmark tests.
People |
ORCID iD |
Amanda Prorok (Principal Investigator) |
Publications
Zheng H
(2020)
An Adversarial Approach to Private Flocking in Mobile Robot Teams
in IEEE Robotics and Automation Letters
Wang B
(2020)
Mobile Robot Path Planning in Dynamic Environments Through Globally Guided Reinforcement Learning
in IEEE Robotics and Automation Letters
Ryan Kortvelesy
(2021)
ModGNN: Expert Policy Approximation in Multi-Agent Systems with a Modular GNN Framework
Rupert Mitchell
(2020)
Multi-Vehicle Mixed-Reality Reinforcement Learning for Autonomous Multi-Lane Driving
Raymond A
(2019)
Culture-Based Explainable Human-Agent Deconfliction
Raymond A
(2022)
Agree to Disagree: Subjective Fairness in Privacy-Restricted Decentralised Conflict Resolution.
in Frontiers in robotics and AI
Qingbiao Li
(2020)
Graph Neural Networks for Multi-Agent Path Planning
Prorok A.
(2022)
The Holy Grail of Multi-Robot Planning: Learning to Generate Online-Scalable Solutions from Offline-Optimal Experts
in Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS
Prorok A
(2020)
Robust Assignment Using Redundant Robots on Transport Networks With Uncertain Travel Time
in IEEE Transactions on Automation Science and Engineering
Mitchell R.
(2020)
Multi-vehicle mixed reality reinforcement learning for autonomous multi-lane driving
in Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS
Matthew Le Maitre
(2020)
Effects of Controller Heterogeneity on Autonomous Vehicle Traffic
Li Q.
(2020)
Graph neural networks for decentralized path planning
in Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS
Li Q
(2021)
Message-Aware Graph Attention Networks for Large-Scale Multi-Robot Path Planning
in IEEE Robotics and Automation Letters
Jan Blumenkamp
(2020)
The Emergence of Adversarial Communication in Multi-Agent Reinforcement Learning
Jan Blumenkamp
(2022)
A Framework for Real-World Multi-Robot Systems Running Decentralized GNN-Based Policies
Gielis J
(2021)
Improving 802.11p for Delivery of Safety-Critical Navigation Information in Robot-to-Robot Communication Networks
in IEEE Communications Magazine
Gama F
(2022)
Synthesizing Decentralized Controllers With Graph Neural Networks and Imitation Learning
in IEEE Transactions on Signal Processing
Blumenkamp J
(2020)
The Emergence of Adversarial Communication in Multi-Agent Reinforcement Learning
Alex Raymond
(2020)
Culture-Based Explainable Human-Agent Deconfliction
Description | At this stage of the project, there are already 4 key contributions and findings: (1) We developed a foundational algorithms (adversarial co-optimization for robust formation control as well as new inter-robot networking techniques). These algorithms will provide a new basis for the development of resilient control policies in multi-agent formations. (2) We developed and successfully tested a new framework (mixed-reality for online reinforcement learning). This framework will enable more aggressive algorithm development towards resilient autonomous systems. (3) We developed a new, breakthrough neural architecture for autonomous agent coordination. This is the first time GNNs are used for learning coordination of robots. (4) We developed a software framework that allows for real world deployment of the policies learnt in step (3). (5) We performed an important human experiment to validate the importance of explainability in human-robot deconfliction. This ultimately leads to more resilient, trustworthy human-robot communities. Several YouTube movies that illustrate the above work are available on my website. |
Exploitation Route | Our results consist of novel, foundational algorithms (points 1-4 above), as well as important empirical insights (point 5 above). At this stage, the main beneficiaries are academia. My lab has already been contacted by industry, who are very keen to adopt our work. I foresee significant impact in industry and society in the non-distant future. |
Sectors | Aerospace, Defence and Marine,Digital/Communication/Information Technologies (including Software),Transport |
URL | http://www.proroklab.org |
Description | Industry: Ongoing collaborations include Arm (who are sponsoring my research), and Amazon (who have recently awarded me with a research award). I have further been contacted by several companies (in the domain of automation) and who are keen to adopt our Graph Neural Network techniques for robust robot scheduling and coordination in factory settings and navigation. It is still early in the process, but I expect several substantial technical collaborations to result of this interest. Society: I have been invited to speak at large, general audience events (Thinking Digital Conference and Science Festivals); my work has also been featured in the BBC (to be aired in April 2020). This will educate society about the work we are doing. I also expect this to initiate a two-way dialogue which will influence the approaches we are taking. |
First Year Of Impact | 2019 |
Sector | Construction,Digital/Communication/Information Technologies (including Software),Education,Manufacturing, including Industrial Biotechology,Transport |
Impact Types | Cultural |
Description | I am Associate Editor for multiple top-tier publication venues. |
Geographic Reach | Multiple continents/international |
Policy Influence Type | Membership of a guideline committee |
Impact | I am active in three key roles: IEEE Transactions on Robotics and Automation (T-RO), Editor of Special Issue on "Resilience for Networked Robotic Systems"; IEEE Robotics and Automation Letters (R-AL) Associate Editor, and Autonomous Robots Journal (AURO), Associate Editor, I am responsible for the publication of research related to robotics and autonomous systems. |
Description | Training of postdocs |
Geographic Reach | Local/Municipal/Regional |
Policy Influence Type | Influenced training of practitioners or researchers |
Impact | I trained two postdocs with this award. One of these postdocs is now in the process of finalizing a new academic faculty appointment, and the other postdoc is in the process of applying to faculty positions. |
Description | Amazon Research Awards |
Amount | $67,000 (USD) |
Organisation | Amazon.com |
Sector | Private |
Country | United States |
Start | 03/2019 |
Description | Co-Evolving Built Environments and Mobile Autonomy for Future Transport and Mobility |
Amount | £75,000 (GBP) |
Funding ID | InnovateUK grant number RG96233 |
Organisation | Digital Built Britain |
Sector | Private |
Country | United Kingdom |
Start | 09/2018 |
End | 07/2019 |
Description | Donation from Arm |
Amount | £85,000 (GBP) |
Organisation | Arm Limited |
Sector | Private |
Country | United Kingdom |
Start | 07/2019 |
Description | EPSRC capital award for core equipment |
Amount | £16,000 (GBP) |
Funding ID | G108473 |
Organisation | United Kingdom Research and Innovation |
Sector | Public |
Country | United Kingdom |
Start | 11/2020 |
End | 04/2022 |
Description | Resilient Multi-Sensor Fusion Through GNN Based Information Aggregation |
Amount | £225,564 (GBP) |
Organisation | Huawei Technologies |
Sector | Private |
Country | China |
Start | 04/2021 |
End | 05/2022 |
Description | Scalable Co-optimization of Collective Robotic Mobility and the Artificial Environment |
Amount | € 1,500,000 (EUR) |
Organisation | European Research Council (ERC) |
Sector | Public |
Country | Belgium |
Start | 01/2021 |
End | 12/2026 |
Description | Studentship - Nokia Bell Labs. |
Amount | £63,000 (GBP) |
Organisation | Nokia Research Centre Cambridge |
Sector | Private |
Country | United Kingdom |
Start | 09/2019 |
End | 02/2023 |
Description | Collaboration with Montreal Polytechnique: visiting researcher, Jacopo Panerati |
Organisation | École Polytechnique de Montréal |
Country | Canada |
Sector | Academic/University |
PI Contribution | I hosted the visiting researcher Jacopo Panerati in my team. I supervised and mentored Jacopo, and together, we supervised several of my students. This research collaboration resulted in two publications. |
Collaborator Contribution | My colleague at Montreal Polytechnique sent over a visiting researcher, Jacopo Panerati, who took up the role of a PDRA whilst he was with us in the summer of 2019. Jacopo developed research under my direction; he guided my students on a daily basis. |
Impact | We published two papers jointly: one journal paper (IEEE R-AL) and one conference paper (AAMAS). We are working on a third manuscript. |
Start Year | 2019 |
Description | Collaboration with University of Pennsylvania, USA |
Organisation | University of Pennsylvania |
Country | United States |
Sector | Academic/University |
PI Contribution | I initiated the collaboration, and invited two colleagues from UPenn to contribute to our research thrust; the three of us supported my Phd student, Qingbiao Li, who is the lead author on the resulting paper. We developed a novel method to solve the path planning problem in a decentralized way, by way of leveraging a novel learning architecture (graph neural networks). |
Collaborator Contribution | Fernando Gama and Alejandro Ribeiro (both from UPenn, USA) are experts in 'aggregation graph neural networks'. We together established the methodology by which to apply this to decentralized multi-agent path planning. |
Impact | We have published one paper, and a second paper is currently under review. Our GNN architecture has shown to be very effective at solving the multi-agent path planning problem. We were able to devise a very efficient training scheme and produced state-of-the art results. My partners from UPenn are experts in the domain of Signal Processing, and complement our knowledge in this domain. |
Start Year | 2019 |
Description | Collaboration with interdisciplinary faculty member in Department of Computer Science and Technology |
Organisation | University of Cambridge |
Country | United Kingdom |
Sector | Academic/University |
PI Contribution | I supervised my PhD student Alex Raymond, guiding him on a daily/weekly basis. I initiated the research thrust and taught him methodology on multi-agent path planning. |
Collaborator Contribution | Hatice Gunes provided guidance on the human experiment design and execution. |
Impact | We have published a paper at AAMAS 2020. |
Start Year | 2019 |
Title | Method and System for Robot Navigation in Unknown Environments |
Description | The present techniques generally relate to a method and system for robot navigation in an unknown environment. In particular, the present techniques provide a method for training a machine learning, ML, model for enabling a robot or navigating device to navigate through an unknown environment to a target object using input from a network of sensors, and a navigation system that uses a trained ML model to guide the robot/navigating device to a target object. |
IP Reference | 2106286.4 |
Protection | Patent application published |
Year Protection Granted | 2022 |
Licensed | No |
Impact | This application is the seed of a spin-off opportunity and further commercially-minded seed funds. |
Title | Advesarial Communications through Reinforcement Learning based on GNNs |
Description | Adversarial Communications through Reinforcement Learning based on GNNs |
Type Of Technology | Software |
Year Produced | 2020 |
Open Source License? | Yes |
Impact | Adversarial Communications through Reinforcement Learning based on GNNs |
Title | Decentralized multi-agent control using GNNs for ICRA 2022 |
Description | Github repository of open source code that enables the deployment of ROS2 based multi-robot coordination. |
Type Of Technology | Software |
Year Produced | 2022 |
Open Source License? | Yes |
Impact | Developers are forking our repo. |
Title | Github open-source code for adversarial co-optimization of formation control. |
Description | Github repository of code developed to produce one of our academic papers. In particular, we provide the code for the adversarial co-optimization procedure use to optimize formation control in multi-vehicle systems. Reference: An Adversarial Approach to Private Flocking in Mobile Robot Teams Hehui Zheng (1), Jacopo Panerati (2), Giovanni Beltrame (2), Amanda Prorok (1) ((1) University of Cambridge, (2) Polytechnique Montreal) arXiv: 1909.10387. |
Type Of Technology | Software |
Year Produced | 2019 |
Open Source License? | Yes |
Impact | The release is very recent, hence, impact has yet to be measured. |
URL | https://github.com/proroklab/private_flocking |
Description | Computer Science Residential for Women |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Schools |
Results and Impact | I hosted a 3-hour long robot programming workshop for group of 30 school girls. |
Year(s) Of Engagement Activity | 2019 |
Description | Invited Seminar - Oxford Control Group |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Schools |
Results and Impact | Invited seminar at Oxford University Control Group. Roughly 60 people attended, a mix of faculty, postdocs, graduates and undergraduates. This sparked a vigorous discussion (indeed, my seminar ran 15 minutes over the 1-hour allotted time!). There is a lot of excitement about my work. I also prospect future collaborations with Oxford to emerge due to this visit. |
Year(s) Of Engagement Activity | 2020 |
Description | Invited Seminar - UKRI Trustworthy Autonomous Systems (TAS) Node in Resilience |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | Regional |
Primary Audience | Schools |
Results and Impact | UKRI Trustworthy Autonomous Systems (TAS) Node in Resilience 04/2021 Invited speaker at seminar series. Invited by Prof. Radu Calinescu. "Resilient and Robust Coordination in Multi-Agent Systems" |
Year(s) Of Engagement Activity | 2021 |
URL | https://resilience.tas.ac.uk |
Description | Invited Seminar, Cornell University, Robotics Seminar Series |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | Regional |
Primary Audience | Postgraduate students |
Results and Impact | Invited Seminar talk given to Cornell's robotics researchers. |
Year(s) Of Engagement Activity | 2020 |
Description | Keynote - UK-Robotics and Automation Society - Conference |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Keynote seminar at yearly UK-RAS conference (UK Robotics and Automation Society). |
Year(s) Of Engagement Activity | 2019 |
Description | Keynote Talk - IEEE International Conference on Unmanned Systems, 2020 |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Plenary speaker. Invited by Prof. Jie Chen. "Learning Communication for Decentralized Coordination in Multi-Agent Systems". |
Year(s) Of Engagement Activity | 2020 |
Description | Keynote presentation at MobiUK 2019 |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Postgraduate students |
Results and Impact | The keynote sparked a lot of interest. It lead to the consolidation of my relationship with Nokia Bell Labs and their funding of a studentship. |
Year(s) Of Engagement Activity | 2019 |
Description | School Visit (Bristol) |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Postgraduate students |
Results and Impact | Invited seminar at robotics seminar series. |
Year(s) Of Engagement Activity | 2019 |
Description | School visit (Herriot-Watt) |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Postgraduate students |
Results and Impact | Invited keynote at Herriot-Watt Robotics Centre Open Day. I also held a second, workshop-like talk about 'being a successful female academic'. |
Year(s) Of Engagement Activity | 2019 |
Description | TV interview and demo for BBC |
Form Of Engagement Activity | A broadcast e.g. TV/radio/film/podcast (other than news/press) |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Public/other audiences |
Results and Impact | BBC came to interview me and made videos of a robot demo that my lab prepared for them. This footage will appear on TV in April, 2020. |
Year(s) Of Engagement Activity | 2019 |