Resilient Path Coordination in Connected Vehicle Systems

Lead Research Organisation: University of Cambridge
Department Name: Computer Laboratory

Abstract

The deployment of connected and autonomous vehicles presents us with transformational opportunities for road transport. As the standardization of inter-vehicular communications progresses, vehicles will soon be wirelessly connected, enabling coordinated driving strategies. By enabling vehicles to jointly agree on optimal maneuvers and navigation strategies, coordinated driving promises to improve overall traffic throughput, road capacity, and passenger safety. However, coordinated driving in connected and autonomous vehicle systems suffers from one key limitation: all vehicles in the system are assumed to be cooperative. This is an issue, since automated vehicle control systems are susceptible to numerous failure conditions, ranging from internally triggered faults (e.g., hardware, software, or communications failures) to externally triggered faults (e.g., environmental disturbances or malicious tampering). When cooperation breaks down due to these faults, coordinated driving strategies lead to negative unanticipated consequences, compromising passenger safety and traffic fluidity.

The goal of this project is to develop a resilient coordinated driving method for multi-vehicle systems that are potentially non-cooperative and unreliable. The issue of providing resilience in the face of non-cooperation (e.g., faulty, byzantine or adversarial agents) has received considerable attention within the domain of distributed network control. However, it is not until very recently that we have started to tackle the question of how to deal with failures and misbehavior when connected agents (vehicles or robots) are mobile. Although preliminary results are promising, the methods deal with control (without planning), and cannot handle discrete workspace constraints (i.e., lane topographies) nor kinodynamic constraints (i.e., car-like motion primitives). Consequently, they do not lend themselves to the problem of coordinated driving. There lies a gap between what we know about resilient network control, and what we know about resilient path coordination. The main contribution of this research programme is to fill this gap by providing methods for resilient planning of trajectories for car-like vehicles with lane constraints.

Our methodology is based on a juxtaposition of centralized and decentralized path planning: we will leverage decentralized planning to guarantee collision-free paths at all times and in all circumstances, and couple this with centralized planning that optimizes global objectives whenever possible. The aim is to develop an adaptive algorithm that slides between the distinct modes as a function of real-time factors that define the level of cooperation in the multi-vehicle system. Such sliding mode architectures have yet to be established within the context of connected multi-vehicle systems, where vehicles cannot be assumed to be cooperative at all times. The proposed research will build upon the expertise of the PI in the field of resilient control, inter-vehicular coordination, and optimization.

The implications of this research are expected to contribute to the theory of multi-vehicle path planning and control, with direct applications to connected and autonomous vehicles. This will ultimately contribute to the improvement of future road transport systems, addressing both safety as well as efficiency.

Planned Impact

The real-world installment of V2V and V2I infrastructure is still in a preliminary stage - we have a window of opportunity to develop reliable and resilient coordinated driving solutions, before large-scale deployment of connected (and autonomous) vehicles takes place.

The benefits of resilient coordinated driving methods are:
(a) gains in road capacity, traffic throughput, and fuel efficiency, without compromising passenger safety; this will be facilitated by our sliding mode path coordination algorithm;
(b) creation and maintenance of user trust in coordinated driving systems, with direct implications on the commercial aspects of involved industries; this will be facilitated by formal guarantees and empirical validations of our methods;
(c) drive the generation of further knowledge; this will be facilitated through our principled study of failure modes in coordinated multi-vehicle systems, and through the publication of benchmark tests.
 
Description At this stage of the project, there are already 4 key contributions and findings:
(1) We developed a foundational algorithms (adversarial co-optimization for robust formation control as well as new inter-robot networking techniques). These algorithms will provide a new basis for the development of resilient control policies in multi-agent formations.
(2) We developed and successfully tested a new framework (mixed-reality for online reinforcement learning). This framework will enable more aggressive algorithm development towards resilient autonomous systems.
(3) We developed a new, breakthrough neural architecture for autonomous agent coordination. This is the first time GNNs are used for learning coordination of robots.
(4) We developed a software framework that allows for real world deployment of the policies learnt in step (3).
(5) We performed an important human experiment to validate the importance of explainability in human-robot deconfliction. This ultimately leads to more resilient, trustworthy human-robot communities.

Several YouTube movies that illustrate the above work are available on my website.
Exploitation Route Our results consist of novel, foundational algorithms (points 1-4 above), as well as important empirical insights (point 5 above). At this stage, the main beneficiaries are academia. My lab has already been contacted by industry, who are very keen to adopt our work. I foresee significant impact in industry and society in the non-distant future.
Sectors Aerospace, Defence and Marine,Digital/Communication/Information Technologies (including Software),Transport

URL http://www.proroklab.org
 
Description Industry: Ongoing collaborations include Arm (who are sponsoring my research), and Amazon (who have recently awarded me with a research award). I have further been contacted by several companies (in the domain of automation) and who are keen to adopt our Graph Neural Network techniques for robust robot scheduling and coordination in factory settings and navigation. It is still early in the process, but I expect several substantial technical collaborations to result of this interest. Society: I have been invited to speak at large, general audience events (Thinking Digital Conference and Science Festivals); my work has also been featured in the BBC (to be aired in April 2020). This will educate society about the work we are doing. I also expect this to initiate a two-way dialogue which will influence the approaches we are taking.
First Year Of Impact 2019
Sector Construction,Digital/Communication/Information Technologies (including Software),Education,Manufacturing, including Industrial Biotechology,Transport
Impact Types Cultural

 
Description I am Associate Editor for multiple top-tier publication venues.
Geographic Reach Multiple continents/international 
Policy Influence Type Membership of a guideline committee
Impact I am active in three key roles: IEEE Transactions on Robotics and Automation (T-RO), Editor of Special Issue on "Resilience for Networked Robotic Systems"; IEEE Robotics and Automation Letters (R-AL) Associate Editor, and Autonomous Robots Journal (AURO), Associate Editor, I am responsible for the publication of research related to robotics and autonomous systems.
 
Description Training of postdocs
Geographic Reach Local/Municipal/Regional 
Policy Influence Type Influenced training of practitioners or researchers
Impact I trained two postdocs with this award. One of these postdocs is now in the process of finalizing a new academic faculty appointment, and the other postdoc is in the process of applying to faculty positions.
 
Description Amazon Research Awards
Amount $67,000 (USD)
Organisation Amazon.com 
Sector Private
Country United States
Start 03/2019 
 
Description Co-Evolving Built Environments and Mobile Autonomy for Future Transport and Mobility
Amount £75,000 (GBP)
Funding ID InnovateUK grant number RG96233 
Organisation Digital Built Britain 
Sector Private
Country United Kingdom
Start 09/2018 
End 07/2019
 
Description Donation from Arm
Amount £85,000 (GBP)
Organisation Arm Limited 
Sector Private
Country United Kingdom
Start 07/2019 
 
Description EPSRC capital award for core equipment
Amount £16,000 (GBP)
Funding ID G108473   
Organisation United Kingdom Research and Innovation 
Sector Public
Country United Kingdom
Start 11/2020 
End 04/2022
 
Description Resilient Multi-Sensor Fusion Through GNN Based Information Aggregation
Amount £225,564 (GBP)
Organisation Huawei Technologies 
Sector Private
Country China
Start 04/2021 
End 05/2022
 
Description Scalable Co-optimization of Collective Robotic Mobility and the Artificial Environment
Amount € 1,500,000 (EUR)
Organisation European Research Council (ERC) 
Sector Public
Country Belgium
Start 01/2021 
End 12/2026
 
Description Studentship - Nokia Bell Labs.
Amount £63,000 (GBP)
Organisation Nokia Research Centre Cambridge 
Sector Private
Country United Kingdom
Start 09/2019 
End 02/2023
 
Description Collaboration with Montreal Polytechnique: visiting researcher, Jacopo Panerati 
Organisation École Polytechnique de Montréal
Country Canada 
Sector Academic/University 
PI Contribution I hosted the visiting researcher Jacopo Panerati in my team. I supervised and mentored Jacopo, and together, we supervised several of my students. This research collaboration resulted in two publications.
Collaborator Contribution My colleague at Montreal Polytechnique sent over a visiting researcher, Jacopo Panerati, who took up the role of a PDRA whilst he was with us in the summer of 2019. Jacopo developed research under my direction; he guided my students on a daily basis.
Impact We published two papers jointly: one journal paper (IEEE R-AL) and one conference paper (AAMAS). We are working on a third manuscript.
Start Year 2019
 
Description Collaboration with University of Pennsylvania, USA 
Organisation University of Pennsylvania
Country United States 
Sector Academic/University 
PI Contribution I initiated the collaboration, and invited two colleagues from UPenn to contribute to our research thrust; the three of us supported my Phd student, Qingbiao Li, who is the lead author on the resulting paper. We developed a novel method to solve the path planning problem in a decentralized way, by way of leveraging a novel learning architecture (graph neural networks).
Collaborator Contribution Fernando Gama and Alejandro Ribeiro (both from UPenn, USA) are experts in 'aggregation graph neural networks'. We together established the methodology by which to apply this to decentralized multi-agent path planning.
Impact We have published one paper, and a second paper is currently under review. Our GNN architecture has shown to be very effective at solving the multi-agent path planning problem. We were able to devise a very efficient training scheme and produced state-of-the art results. My partners from UPenn are experts in the domain of Signal Processing, and complement our knowledge in this domain.
Start Year 2019
 
Description Collaboration with interdisciplinary faculty member in Department of Computer Science and Technology 
Organisation University of Cambridge
Country United Kingdom 
Sector Academic/University 
PI Contribution I supervised my PhD student Alex Raymond, guiding him on a daily/weekly basis. I initiated the research thrust and taught him methodology on multi-agent path planning.
Collaborator Contribution Hatice Gunes provided guidance on the human experiment design and execution.
Impact We have published a paper at AAMAS 2020.
Start Year 2019
 
Title Method and System for Robot Navigation in Unknown Environments 
Description The present techniques generally relate to a method and system for robot navigation in an unknown environment. In particular, the present techniques provide a method for training a machine learning, ML, model for enabling a robot or navigating device to navigate through an unknown environment to a target object using input from a network of sensors, and a navigation system that uses a trained ML model to guide the robot/navigating device to a target object. 
IP Reference 2106286.4 
Protection Patent application published
Year Protection Granted 2022
Licensed No
Impact This application is the seed of a spin-off opportunity and further commercially-minded seed funds.
 
Title Advesarial Communications through Reinforcement Learning based on GNNs 
Description Adversarial Communications through Reinforcement Learning based on GNNs 
Type Of Technology Software 
Year Produced 2020 
Open Source License? Yes  
Impact Adversarial Communications through Reinforcement Learning based on GNNs 
 
Title Decentralized multi-agent control using GNNs for ICRA 2022 
Description Github repository of open source code that enables the deployment of ROS2 based multi-robot coordination. 
Type Of Technology Software 
Year Produced 2022 
Open Source License? Yes  
Impact Developers are forking our repo. 
 
Title Github open-source code for adversarial co-optimization of formation control. 
Description Github repository of code developed to produce one of our academic papers. In particular, we provide the code for the adversarial co-optimization procedure use to optimize formation control in multi-vehicle systems. Reference: An Adversarial Approach to Private Flocking in Mobile Robot Teams Hehui Zheng (1), Jacopo Panerati (2), Giovanni Beltrame (2), Amanda Prorok (1) ((1) University of Cambridge, (2) Polytechnique Montreal) arXiv: 1909.10387. 
Type Of Technology Software 
Year Produced 2019 
Open Source License? Yes  
Impact The release is very recent, hence, impact has yet to be measured. 
URL https://github.com/proroklab/private_flocking
 
Description Computer Science Residential for Women 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Schools
Results and Impact I hosted a 3-hour long robot programming workshop for group of 30 school girls.
Year(s) Of Engagement Activity 2019
 
Description Invited Seminar - Oxford Control Group 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Schools
Results and Impact Invited seminar at Oxford University Control Group. Roughly 60 people attended, a mix of faculty, postdocs, graduates and undergraduates. This sparked a vigorous discussion (indeed, my seminar ran 15 minutes over the 1-hour allotted time!). There is a lot of excitement about my work. I also prospect future collaborations with Oxford to emerge due to this visit.
Year(s) Of Engagement Activity 2020
 
Description Invited Seminar - UKRI Trustworthy Autonomous Systems (TAS) Node in Resilience 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Schools
Results and Impact UKRI Trustworthy Autonomous Systems (TAS) Node in Resilience 04/2021 Invited speaker at seminar series. Invited by Prof. Radu Calinescu. "Resilient and Robust Coordination in Multi-Agent Systems"
Year(s) Of Engagement Activity 2021
URL https://resilience.tas.ac.uk
 
Description Invited Seminar, Cornell University, Robotics Seminar Series 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Postgraduate students
Results and Impact Invited Seminar talk given to Cornell's robotics researchers.
Year(s) Of Engagement Activity 2020
 
Description Keynote - UK-Robotics and Automation Society - Conference 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Keynote seminar at yearly UK-RAS conference (UK Robotics and Automation Society).
Year(s) Of Engagement Activity 2019
 
Description Keynote Talk - IEEE International Conference on Unmanned Systems, 2020 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Plenary speaker. Invited by Prof. Jie Chen. "Learning Communication for Decentralized Coordination in Multi-Agent Systems".
Year(s) Of Engagement Activity 2020
 
Description Keynote presentation at MobiUK 2019 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact The keynote sparked a lot of interest. It lead to the consolidation of my relationship with Nokia Bell Labs and their funding of a studentship.
Year(s) Of Engagement Activity 2019
 
Description School Visit (Bristol) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Postgraduate students
Results and Impact Invited seminar at robotics seminar series.
Year(s) Of Engagement Activity 2019
 
Description School visit (Herriot-Watt) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Postgraduate students
Results and Impact Invited keynote at Herriot-Watt Robotics Centre Open Day.
I also held a second, workshop-like talk about 'being a successful female academic'.
Year(s) Of Engagement Activity 2019
 
Description TV interview and demo for BBC 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact BBC came to interview me and made videos of a robot demo that my lab prepared for them. This footage will appear on TV in April, 2020.
Year(s) Of Engagement Activity 2019