Swarming algorithms to achieve goal based missions
Lead Research Organisation:
Cranfield University
Department Name: Sch of Aerospace, Transport & Manufact
Abstract
Using swarms to conduct missions is not a new topic and a number of companies are looking at the application of large numbers of drones to achieve these missions. Such swarms are typically controlled using a centralised system such as a ground station or manned aircraft/ vehicle, and this is not considered as exploiting true swarm capabilities due to the command structure. The advantage of emergent swarm behaviours is to overcome the excessive operator Command and Control manpower utilisation, and to become resilient, and not reliant upon hierarchical, single point of failure control nodes. The proposed research is to identify the characteristics which determine the level of intelligence, and inter unit communications, necessary to establish and affect different emergent behaviours from the overall swarm. The research will identify how different emergent behaviours could be derived from a parametrised simple deployed algorithm.
Studentship Projects
| Project Reference | Relationship | Related To | Start | End | Student Name |
|---|---|---|---|---|---|
| EP/V519509/1 | 01/10/2020 | 30/09/2027 | |||
| 2454254 | Studentship | EP/V519509/1 | 01/10/2020 | 30/09/2024 | Marc Espinos Longa |
| Description | The points mentioned below are related to the development of multi-agent swarm systems to achieve goal-based missions in a decentralised fashion, comprehending from high-level deep reinforcement learning control, to hybrid human-machine teams through bio-inspired techniques. >> Bio-inspired Swarm Intelligence in Safe and Rescue Operations (Journal of Intelligent & Robotic Systems: https://doi.org/10.1007/s10846-022-01690-5) - within a high-level 2D simulation of a hypothetic post-earthquake scenario, the designed decentralised systems combine bacterial and honey bee foraging behaviours on teams of UAVs to locate victims and broadcast their positions to human rescue teams. Both systems demonstrate to be scalable and robust on scouting tasks under poor communication schemes and highly uncertain environments. >> Swarm Intelligence in Cooperative Environments: N-Step Dynamic Tree Search Algorithm (Journal of Aerospace Information Systems: under peer-review, American Control Conference 2022: https://doi.org/10.23919/ACC53348.2022.9867171, AIAA SciTech 2022 https://doi.org/10.2514/6.2022-1839) - decentralised multi-agent reinforcement learning (RL) algorithm design inspired in tree search. The system uses forward planning (using a trained environment model) and direct RL updates to learn faster and enhance performance. The N-Step Dynamic Tree Search aims to adapt single-agent tree search learning methods to the multi-agent boundaries, and demonstrates to be a remarkable advance compared to conventional RL techniques. >> Decentralised Multi-Agent Deep Reinforcement Learning: A Competitive-Game Perspective (Nature Scientific Reports: under peer-review, preprint: https://doi.org/10.21203/rs.3.rs-2065000/v1) - this work makes a deep analysis of multi-agent and decentralised deep reinforcement learning (DRL) algorithms applied to a competitive game where two team of agents need to compete to win a couple of simulation-based games. The manuscript studies not only different algorithms (e.g., Q-based and actor-critic methods) but also different system architectures, i.e., data storing and sampling strategies, team-shared networks, etc. Results indicate that Q-based methods perform better than actor-critic approaches with local environment observations and no inter-agent communication. >> Drone Deliveries in Controlled Environments (upcoming article) - test and validation of decentralised multi-agent Q-based DRL techniques in a simulated and real multi-agent drone delivery indoor flight arena (the warehouse problem). The implemented system combines a decentralised and generalised DRL network (trained in sofware in the loop) for high-level task control and a low-level potential field controller for dynamic obstacle avoidance. The heterogeneous team of UAVs proves to complete their respective delivery tasks in both simulation and real environments. |
| Exploitation Route | The presented research is useful for: >> People (industry and academics) working in the field of robotics/aerospace/defence and marine in problems related to multi-agent autonomous area coverage (e.g., safe and rescue field, surveillance, geophysical mapping, security, planetary exploration). >> Researchers in the field of multi-agent reinforcement learning (RL) and deep reinforcement learning (DRL) that are willing to test and evaluate tree search techniques. Projects that aim to build robust multi-agent DRL systems with limited communication schemes and heterogeneous teaming (e.g., UAV high-level control). Aside from the aforementioned, this research exposes interesting model training methodologies and architectures, as well as the integration of DRL models in traditional control systems. Thus, my research is likely to be useful for other industries/research fields that are willing to incorporate artificial intelligence (AI) and DRL in their solutions, seeking to improve performance |
| Sectors | Aerospace, Defence and Marine,Digital/Communication/Information Technologies (including Software) |
| URL | https://www.cranfield.ac.uk/people/marc-espins-longa-28210423 |
| Description | BAE Systems Virtual PhD Conference |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Industry/Business |
| Results and Impact | The intended purpose of this presentation was to showcase the on-going research of a selection of PhD researchers sponsored by BAE Systems to the company and industry related partners. The audience demonstrated to be interested and satisfied with the work carried out since the beginning of the award. BAE Systems presented a patent application from that peace of research, which is currently under review. |
| Year(s) Of Engagement Activity | 2021 |
| Description | Cranfield Aerospace Research Student Conference 2022 |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | Local |
| Primary Audience | Industry/Business |
| Results and Impact | CARSC22 (hybrid conference) is the Aerospace Theme's annual opportunity to showcase the latest advances of every research centre of Cranfield Aerospace. I had the pleasure to represent one of the research centres (Centre for Autonomous and Cyber-Physical Systems) and be part of the expert panel in charge of evaluating the exposed research content. Four external plneary speakers (working at Scoring Group plc, Thales UK, Rolls Royce and The Manufacturing Technology Centre Ltd) were invited to give talks and participate in a debate lead by the academic lead for the aerospace thematic doctoral community and the former panel. The audience demonstrated real involvement, interest, and curiosity, asking both technical and broader questions related with industry and research activities. |
| Year(s) Of Engagement Activity | 2022 |
| URL | https://www.cranfield.ac.uk/events/events-2022/cranfield-aerospace-research-student-conference-2022?... |
| Description | Research Conference |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Industry/Business |
| Results and Impact | Presented the first part of the study "Swarm Intelligence in Cooperative Environments: N-Step Dynamic Tree Search Algorithm" at the 2022 AIAA SciTech Conference (San Diego, California, USA) Around 20 to 50 people from industry/academy attended to the respective talk showcasing the results of the ongoing reinforcement learning work. The audience showed deep interest in the implemented system, asking questions about the range of application, and technical questions related to system robsutness, scalability, and system architecture. |
| Year(s) Of Engagement Activity | 2022 |
| URL | https://arc.aiaa.org/doi/10.2514/6.2022-1839 |
| Description | Research Conference |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Industry/Business |
| Results and Impact | Presented the second part of the study "Swarm Intelligence in Cooperative Environments: N-Step Dynamic Tree Search Algorithm" at the 2022 American Control Conference (Atlanta, Georgie, USA). Around 20 to 40 people attended to the respective talk where I showcased the ongoing research related to tree search reinforcement learning applied to decentralised multi-agent decision-making. The audience showed interest and asked for technical questions related to the tree search learning mechanisms involved, and the multi-agent system architecture. |
| Year(s) Of Engagement Activity | 2022 |
| URL | https://ieeexplore.ieee.org/document/9867171 |
| Description | UAV multi-agent warehouse demonstration |
| Form Of Engagement Activity | Participation in an open day or visit at my research institution |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Industry/Business |
| Results and Impact | As part of the Swedish research visit to Cranfield Universiy to enhance aerospace, aviation, space, and defence academic and business relationships, we (a postdoctoral researcher and myself) carried out a real demonstration of our UAV multi-agent warehouse experiment. The guests responded with great enthusiasm and interest, asking questions about the implemented DRL system, the hardware used, communications, etc. |
| Year(s) Of Engagement Activity | 2023 |
| URL | https://www.linkedin.com/posts/marc-espinos-longa_phd-engineering-warehousing-activity-7036651502962... |