ACE-OPS: From Autonomy to Cognitive assistance in Emergency OPerationS
Lead Research Organisation:
University of Oxford
Department Name: Computer Science
Abstract
The vision of this collaborative multi-centre project is to safeguard and transform current operation protocols of emergency teams by providing sensing, situation awareness, cognitive assistance and mobile autonomy capabilities working synergistically as a single system. Statistics collected by the Home Office report 346 fire related fatalities in England during 2016/2017, the highest figure since 2011/12. Over a 10-year period in USA, 2775 firefighters died on duty. Where there is a need to save and evacuate people from a burning or flooded building, it is important for the chief incident commander to have increased situational awareness and to be able to effectively coordinate the rescue operation, and for individual responders to have enhanced visibility of surrounding hazards and dangers. To this end, we need to combine UK-based expertise in mobile autonomy and people localisation, with internationally leading expertise on welfare monitoring and cognitive assistance at the Univ. of Virginia, and on robotic vision applied to aerial vehicles at the Queensland University of Technology.
The proposed work involves four distinct research directions: 1) providing an integrated system for situation awareness that involves localisation of the emergency responders, monitoring of their welfare and mapping of the dynamically changing environment; 2) exploring how situation awareness information should be fed into cognitive assistance tools, in order to provide helpful triggers and alerts to the incident commander and their team; 3) introducing various levels of autonomy enabling aerial vehicles to simultaneously perform tasks of mapping, communication and localisation; and 4) integrating the above capabilities and building the first end-to-end response system that implements the full feedback loop from sensor acquisition to emergency responders and back to sensor actuation. Sensors on people's wearable devices together with sensors mounted on aerial vehicles will contribute to data acquisition for welfare, location and environment monitoring. This in turn will provide input to cognitive assistance for emergency response teams, helping them to assess the situation. They will then in turn provide feedback to sensor systems to prioritise monitoring of specific areas, people or tasks, thus dynamically influencing the next round of situation awareness, and so on. This feedback loop will be a step change providing a whole new approach to safety for emergency responders.
The proposed work involves four distinct research directions: 1) providing an integrated system for situation awareness that involves localisation of the emergency responders, monitoring of their welfare and mapping of the dynamically changing environment; 2) exploring how situation awareness information should be fed into cognitive assistance tools, in order to provide helpful triggers and alerts to the incident commander and their team; 3) introducing various levels of autonomy enabling aerial vehicles to simultaneously perform tasks of mapping, communication and localisation; and 4) integrating the above capabilities and building the first end-to-end response system that implements the full feedback loop from sensor acquisition to emergency responders and back to sensor actuation. Sensors on people's wearable devices together with sensors mounted on aerial vehicles will contribute to data acquisition for welfare, location and environment monitoring. This in turn will provide input to cognitive assistance for emergency response teams, helping them to assess the situation. They will then in turn provide feedback to sensor systems to prioritise monitoring of specific areas, people or tasks, thus dynamically influencing the next round of situation awareness, and so on. This feedback loop will be a step change providing a whole new approach to safety for emergency responders.
Planned Impact
First responders operate in extremely hazardous and dangerous conditions, risking their lives and wellbeing to save others. In the UK, 14 firefighters lost their lives over the ten year period 2003-2014. Internationally, over the same period, 2775 firefighters died in the USA.
The ambulance and police services similarly face hostile and perilous working conditions. The challenges faced by first responders motivates the need for an urgent and internationally guided effort to improve the state-of-the-art in terms of situational awareness, reducing first responder deaths and injuries. In turn, better operating knowledge translates to improved outcomes for victim rescue, impacting the broader public through faster and more targetted responses. This has widespread social and welfare benefits. To achieve these goals, our proposed project includes strong support from Dstl and the Home Office to incorporate technological advances into first responder practice in the UK. More broadly, they will also support the project through exposure at the International Forum to Advance First Responder Innovation (IFAFRI) which links Australia, Canada, the European Commission, Finland, Germany, Israel, Japan, Mexico, New Zealand, Singapore, Spain, Sweden, The Netherlands, United Kingdom, and the United States. Through existing partnerships, we have strong links with the National Fire Chiefs Council (NFCC) which is the umbrella organization for informing government and formulating best operating practice and guidance. At a regional level, we have an ongoing relationship with the Hampshire Fire and Rescue Service, whose facilities and expertise we will use for testing. Internationally, in the United States, the National Institute of Standards and Technology is leading the Public Safety Innovation Accelerator Program which seeks to transform first responder protocol and safety through advanced technology. Trigoni and Stakonovic both lead grants funded by NIST and this provides an ideal mechanism for widespread impact to the first responder community in the United States. Through NIST, UVA and its associated stakeholder conferences, we will disseminate key findings of this project and work closely with US services. We will work with these and other partners to inform technology design, test and refine implementations and create a novel platform that will provide the first end-to-end first responder technology for cognitive, autonomous and enhanced situational awareness and control. The ultimate successful impact of this project will be the adoption of the designed technology which leads to a reduction in fatalities and injuries, both to first responders and the general public.
In terms of economic impact, improved situational awareness and response can lead to better management of disasters and incidents. Benefits of improved safety could lead to lower first responder time loss due to hospitalization etc. Better monitoring of welfare can also lead to more targeted and informed responses to factors such as dehydration and heat stress. It is also likely that the technology created will generate new intellectual property, with consequent opportunity for licensing, spinout and commercial exploitation.
Workshops will be run at the end of years 2 and 3 to disseminate findings and network with key partners, academic, governmental and from the first responder community. We will also pursue high impact publications in leading conferences and journals e.g. AAAI, IJCAI, ICRA, NIPS, TPAMI etc., to disseminate our novel research.
The national and international importance of this project makes it a high profile demonstration of how the UK research councils are funding state-of-the-art techniques for improving rescue. As such, this Centre-to-Centre project will attract the interest of the wider public, which will be exploited through a dedicated, media-rich website, frequent press releases and feature articles in the popular press.
The ambulance and police services similarly face hostile and perilous working conditions. The challenges faced by first responders motivates the need for an urgent and internationally guided effort to improve the state-of-the-art in terms of situational awareness, reducing first responder deaths and injuries. In turn, better operating knowledge translates to improved outcomes for victim rescue, impacting the broader public through faster and more targetted responses. This has widespread social and welfare benefits. To achieve these goals, our proposed project includes strong support from Dstl and the Home Office to incorporate technological advances into first responder practice in the UK. More broadly, they will also support the project through exposure at the International Forum to Advance First Responder Innovation (IFAFRI) which links Australia, Canada, the European Commission, Finland, Germany, Israel, Japan, Mexico, New Zealand, Singapore, Spain, Sweden, The Netherlands, United Kingdom, and the United States. Through existing partnerships, we have strong links with the National Fire Chiefs Council (NFCC) which is the umbrella organization for informing government and formulating best operating practice and guidance. At a regional level, we have an ongoing relationship with the Hampshire Fire and Rescue Service, whose facilities and expertise we will use for testing. Internationally, in the United States, the National Institute of Standards and Technology is leading the Public Safety Innovation Accelerator Program which seeks to transform first responder protocol and safety through advanced technology. Trigoni and Stakonovic both lead grants funded by NIST and this provides an ideal mechanism for widespread impact to the first responder community in the United States. Through NIST, UVA and its associated stakeholder conferences, we will disseminate key findings of this project and work closely with US services. We will work with these and other partners to inform technology design, test and refine implementations and create a novel platform that will provide the first end-to-end first responder technology for cognitive, autonomous and enhanced situational awareness and control. The ultimate successful impact of this project will be the adoption of the designed technology which leads to a reduction in fatalities and injuries, both to first responders and the general public.
In terms of economic impact, improved situational awareness and response can lead to better management of disasters and incidents. Benefits of improved safety could lead to lower first responder time loss due to hospitalization etc. Better monitoring of welfare can also lead to more targeted and informed responses to factors such as dehydration and heat stress. It is also likely that the technology created will generate new intellectual property, with consequent opportunity for licensing, spinout and commercial exploitation.
Workshops will be run at the end of years 2 and 3 to disseminate findings and network with key partners, academic, governmental and from the first responder community. We will also pursue high impact publications in leading conferences and journals e.g. AAAI, IJCAI, ICRA, NIPS, TPAMI etc., to disseminate our novel research.
The national and international importance of this project makes it a high profile demonstration of how the UK research councils are funding state-of-the-art techniques for improving rescue. As such, this Centre-to-Centre project will attract the interest of the wider public, which will be exploited through a dedicated, media-rich website, frequent press releases and feature articles in the popular press.
Organisations
- University of Oxford (Lead Research Organisation)
- Queensland University of Technology (QUT) (Collaboration)
- University of Virginia (UVa) (Collaboration)
- Rice University (Collaboration)
- UNIVERSITY OF EDINBURGH (Collaboration)
- UNIVERSITY OF OXFORD (Collaboration)
- Cardiff University (Collaboration)
- UNIVERSITY OF LIVERPOOL (Collaboration)
- Monash University (Collaboration)
- Queensland University of Technology (Project Partner)
- BB7 Fire Limited (Project Partner)
- Defence Science and Technology Laboratory (Project Partner)
- Oracle Corporation U K Ltd (Project Partner)
- FLIR Systems AB (Project Partner)
- University of Virginia (Project Partner)
- National Fire Chiefs Council (Project Partner)
Publications
Chen C
(2020)
A survey on deep learning for localization and mapping: Towards the age of spatial machine intelligence
in arXiv:2006.12567
Gammell JD
(2020)
Batch Informed Trees (BIT*): Informed asymptotically optimal anytime search
in International Journal of Robotics Research (IJRR)
Lu C
(2020)
milliEgo
Wei, B
(2021)
iMag+: An Accurate and Rapidly Deployable Inertial Magneto-Inductive SLAM System
in Transactions on Mobile Computing
Wang W
(2021)
RadarLoc: Learning to Relocalize in FMCW Radar
Description | We have addressed many of the challenges linked to autonomous systems for handling multiple, and often conflicting, operational needs in emergency scenarios. More specifically, the key achievements linked to the project's objectives are: 1) novel localisation and scene understanding algorithms and methodologies that are robust to challenging low visibility and smoke-filled environments; 2) unique datasets on indoor scenarios featuring multiple sensors and collected across different mobile platforms 3) robust task and motion planning algorithms for autonomous vehicles 4) open sourced software libraries for planning |
Exploitation Route | Use of open sourced code by other research groups Use of datasets by other research groups Research outcomes could be the basis for companies to provide safer apparatus for emergency responders |
Sectors | Digital/Communication/Information Technologies (including Software) |
Description | AWS Funded Project: Autonomy in Blue Light Emergency Services |
Amount | £330,000 (GBP) |
Organisation | Amazon.com |
Department | Amazon Web Services |
Sector | Private |
Country | United States |
Start | 09/2020 |
End | 04/2024 |
Title | OdomBeyondVision: An Indoor Multi-modal Multi-platform Odometry Dataset Beyond the Visible Spectrum |
Description | An Indoor Multi-modal Multi-platform Odometry Dataset Beyond the Visible Spectrum. This paper presents a multimodal indoor odom- etry dataset, OdomBeyondVision, featuring multiple sensors across the different spectrum and collected with different mobile platforms. Not only does OdomBeyondVision contain the traditional navigation sensors, sensors such as IMUs, mechanical LiDAR, RGBD camera, it also includes several emerging sensors such as the single-chip mmWave radar, LWIR thermal camera and solid-state LiDAR. With the above sensors on UAV, UGV and handheld platforms, we respectively recorded the multimodal odometry data and their movement trajectories in various indoor scenes and different illumination conditions. We release the exemplar radar, radar-inertial and thermal- inertial odometry implementations to demonstrate their results for future works to compare against and improve upon. The full dataset including toolkit and documentation is publicly avail- able at: https://github.com/MAPS-Lab/OdomBeyondVision. Paper describing dataset is here: https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9981865 |
Type Of Material | Database/Collection of data |
Year Produced | 2022 |
Provided To Others? | Yes |
Impact | Use of dataset by other researchers |
URL | https://github.com/MAPS-Lab/OdomBeyondVision |
Title | Thermal, Visual, Lidar Multi-modal dataset |
Description | This dataset consists of trajectories collected by a multimodal (thermal, radar, inertial, visual, lidar) platform. The data was collected both by both handheld and robotic platforms. The majority of datasets with thermal imagery have been collected by autonomous vehicles (self-driving cars), whereas this dataset is from indoor environments (university and college buildings). |
Type Of Material | Database/Collection of data |
Year Produced | 2020 |
Provided To Others? | No |
Impact | This dataset serves as the basis for a number of papers in thermal-inertial odometry from the CPS group. This dataset will be released later this year (2021) as a public dataset. |
Description | Collaboration on robust depth estimation in challenging illumination conditions |
Organisation | Queensland University of Technology (QUT) |
Country | Australia |
Sector | Academic/University |
PI Contribution | Our Oxford CPS group proposed novel depth estimation approaches for challenging illumination conditions (day and night) and co-authored a paper with QUT |
Collaborator Contribution | Our partner provided valuable feedback into the proposed algorithms and helped co-author a paper accepted at CoRL 2022 |
Impact | A published research paper at the 2023 Conference on Robot Learning |
Start Year | 2022 |
Description | Collaboration on self supervised monocular depth estimation in dynamic scenarios |
Organisation | University of Oxford |
Department | Department of Engineering Science |
Country | United Kingdom |
Sector | Academic/University |
PI Contribution | Identifying issues within current solutions, developing novel algorithms/solutions tailored to address current problems, executing experiments using the devised methodology and drafting a research paper. |
Collaborator Contribution | We worked with Jia-Wang Bian, Postdoctoral Researcher at theTorr Vision Group & Active Vision Lab, Department of Engineering Science, University of Oxford, who helped us review and refine the solution based on his previous experience, discuss and share "dataloader" code for partial datasets, specifically CityScape and Dense Depth for Autonomous Driving dataset. We utilize the CityScape subset in our final iteration. |
Impact | Paper (Submitted to Pattern Recognition 27/12/2023): MGDepth: Motion-Guided Cost Volume For Self-Supervised Monocular Depth In Dynamic Scenarios |
Start Year | 2023 |
Description | Collaboration with Nick Pham from the Univ. of Cardiff on bioimpedance sensing for vital sign monitoring. |
Organisation | Cardiff University |
Department | School of Computer Science and Informatics |
Country | United Kingdom |
Sector | Academic/University |
PI Contribution | Our team is contributing to the development of signal processing and machine learning algorithms for processing bioimpedance data to infer health metrics. |
Collaborator Contribution | The Cardiff team is contributing to the hardware design and refinement as well as helping with the design and implementation of the algorithms for inferring health metrics. |
Impact | We are preparing a joint publication. |
Start Year | 2023 |
Description | Collaboration with researchers from other universities to collect a multi modal indoor odometry dataset |
Organisation | University of Edinburgh |
Department | School of Informatics Edinburgh |
Country | United Kingdom |
Sector | Academic/University |
PI Contribution | We contributed to generating and curating data for the odometry dataset, OdomBeyondVision, featuring multiple sensors across the different spectrum and collected with different mobile platforms. Not only does OdomBeyondVision contain the traditional navigation sensors, sensors such as IMUs, mechanical LiDAR, RGBD camera, it also includes several emerging sensors such as the single-chip mmWave radar, LWIR thermal camera and solid-state LiDAR. With the above sensors on UAV, UGV and handheld platforms, we respectively recorded the multimodal odometry data and their movement trajectories in various indoor scenes and different illumination conditions. We release the exemplar radar, radar-inertial and thermal-inertial odometry implementations to demonstrate their results for future works to compare against and improve upon. The full dataset including toolkit and documentation is publicly available at: https://github.com/MAPS-Lab/OdomBeyondVision. |
Collaborator Contribution | Our partners contributed to analysing the dataset and co-authoring the paper that described the dataset. |
Impact | A unique dataset including toolkit and documentation publicly available at: https://github.com/MAPS-Lab/OdomBeyondVision |
Start Year | 2020 |
Description | Collaboration with researchers from other universities to collect a multi modal indoor odometry dataset |
Organisation | University of Liverpool |
Country | United Kingdom |
Sector | Academic/University |
PI Contribution | We contributed to generating and curating data for the odometry dataset, OdomBeyondVision, featuring multiple sensors across the different spectrum and collected with different mobile platforms. Not only does OdomBeyondVision contain the traditional navigation sensors, sensors such as IMUs, mechanical LiDAR, RGBD camera, it also includes several emerging sensors such as the single-chip mmWave radar, LWIR thermal camera and solid-state LiDAR. With the above sensors on UAV, UGV and handheld platforms, we respectively recorded the multimodal odometry data and their movement trajectories in various indoor scenes and different illumination conditions. We release the exemplar radar, radar-inertial and thermal-inertial odometry implementations to demonstrate their results for future works to compare against and improve upon. The full dataset including toolkit and documentation is publicly available at: https://github.com/MAPS-Lab/OdomBeyondVision. |
Collaborator Contribution | Our partners contributed to analysing the dataset and co-authoring the paper that described the dataset. |
Impact | A unique dataset including toolkit and documentation publicly available at: https://github.com/MAPS-Lab/OdomBeyondVision |
Start Year | 2020 |
Description | Collaborative work on robust indoor mapping with low cost mmWave radar |
Organisation | University of Virginia (UVa) |
Country | United States |
Sector | Academic/University |
PI Contribution | My research group (X. Lu, S. Rosa, P. Zhao, B. Wang, C. Chen, A. Markham and N. Trigoni) carried out the bulk of the research work, including the design, implementation and evaluation of milliMap, a single chip millimetre wave radar based indoor mapping system targeted towards low visibility environments. The work started earlier (2019) and initially was funded by the Mobile Autonomy Research Grant (supporting co-authors C. Lu and S. Rosa) and in its latter phase (end 2019- first half of 2020) by ACE-OPS (supporting co-author C. Chen). The work was refined with C. Chen's help in early 2020 following feedback from Mobisys reviewers. |
Collaborator Contribution | Prof. John A Stankovic contributed to research discussions with researchers in my group. The collaboration started earlier in 2018 when he spent his sabbatical at Oxford working with our group (Cyber Physical Systems) and is ongoing until now - albeit with slightly different focus- in the context of ACE-OPS. Prof John Stankovic had frequent discussions with our group, including how to design experiments to validate the proposed indoor mapping method. He also provided feedback on initial drafts of the paper. |
Impact | Paper publication: "See through smoke: robust indoor mapping with low-cost mmWave radar" Proceedings of the 18th International Conference on Mobile Systems, Applications, and Services |
Start Year | 2018 |
Description | Collaborative work on robust indoor mapping with low cost mmWave radar |
Organisation | University of Virginia (UVa) |
Country | United States |
Sector | Academic/University |
PI Contribution | My research group (X. Lu, S. Rosa, P. Zhao, B. Wang, C. Chen, A. Markham and N. Trigoni) carried out the bulk of the research work, including the design, implementation and evaluation of milliMap, a single chip millimetre wave radar based indoor mapping system targeted towards low visibility environments. The work started earlier (2019) and initially was funded by the Mobile Autonomy Research Grant (supporting co-authors C. Lu and S. Rosa) and in its latter phase (end 2019- first half of 2020) by ACE-OPS (supporting co-author C. Chen). The work was refined with C. Chen's help in early 2020 following feedback from Mobisys reviewers. |
Collaborator Contribution | Prof. John A Stankovic contributed to research discussions with researchers in my group. The collaboration started earlier in 2018 when he spent his sabbatical at Oxford working with our group (Cyber Physical Systems) and is ongoing until now - albeit with slightly different focus- in the context of ACE-OPS. Prof John Stankovic had frequent discussions with our group, including how to design experiments to validate the proposed indoor mapping method. He also provided feedback on initial drafts of the paper. |
Impact | Paper publication: "See through smoke: robust indoor mapping with low-cost mmWave radar" Proceedings of the 18th International Conference on Mobile Systems, Applications, and Services |
Start Year | 2018 |
Description | International Collaboration on Deep Odometry Systems on Edge with EKF-LoRa Backend for Real-Time Indoor Positioning |
Organisation | Monash University |
Country | Australia |
Sector | Academic/University |
PI Contribution | - 6 months of time contribution for research of two graduate researchers and two grant PIs (Niki Trigoni and Andrew Markham), including supervision, research discussions, dataset generation, paper writing etc. |
Collaborator Contribution | ~ 3 months of time contribution for joint research between our group at the University of Oxford and partners at the University of Virginia (US), Monash University (Indonesia) and University of Edinburgh (UK) - Discussion of research ideas and results - given their prior experience in the same field - Co-authoring of paper |
Impact | A paper submitted to 1st workshop on cyber physical systems for emergency response. Paper has been accepted and will be presented in May 2022 at the workshop. |
Start Year | 2021 |
Description | International Collaboration on Deep Odometry Systems on Edge with EKF-LoRa Backend for Real-Time Indoor Positioning |
Organisation | University of Edinburgh |
Country | United Kingdom |
Sector | Academic/University |
PI Contribution | - 6 months of time contribution for research of two graduate researchers and two grant PIs (Niki Trigoni and Andrew Markham), including supervision, research discussions, dataset generation, paper writing etc. |
Collaborator Contribution | ~ 3 months of time contribution for joint research between our group at the University of Oxford and partners at the University of Virginia (US), Monash University (Indonesia) and University of Edinburgh (UK) - Discussion of research ideas and results - given their prior experience in the same field - Co-authoring of paper |
Impact | A paper submitted to 1st workshop on cyber physical systems for emergency response. Paper has been accepted and will be presented in May 2022 at the workshop. |
Start Year | 2021 |
Description | International Collaboration on Deep Odometry Systems on Edge with EKF-LoRa Backend for Real-Time Indoor Positioning |
Organisation | University of Virginia (UVa) |
Country | United States |
Sector | Academic/University |
PI Contribution | - 6 months of time contribution for research of two graduate researchers and two grant PIs (Niki Trigoni and Andrew Markham), including supervision, research discussions, dataset generation, paper writing etc. |
Collaborator Contribution | ~ 3 months of time contribution for joint research between our group at the University of Oxford and partners at the University of Virginia (US), Monash University (Indonesia) and University of Edinburgh (UK) - Discussion of research ideas and results - given their prior experience in the same field - Co-authoring of paper |
Impact | A paper submitted to 1st workshop on cyber physical systems for emergency response. Paper has been accepted and will be presented in May 2022 at the workshop. |
Start Year | 2021 |
Description | International Collaboration on Task and Motion Planning |
Organisation | Rice University |
Country | United States |
Sector | Academic/University |
PI Contribution | ESP (Estimation, Search and Planning) group from the Oxford Robotics Institute, University of Oxford, collaborated with researchers from Rice University and from the Jet Propulsion Laboratory California Institute of Technology on novel path planning algorithms for mobile robots. The collaboration led to a joint paper and open source code. |
Collaborator Contribution | Research ideas and co-authors of joint paper. |
Impact | Novel path planning algorithms and methodologies |
Start Year | 2021 |
Title | EIRM* |
Description | Publish path planning algorithm to open source library of planning algorithms |
Type Of Technology | Software |
Year Produced | 2022 |
Open Source License? | Yes |
Impact | Enhancement of know how on planning algorithms, sharing, and opportunities for further advances |
URL | http://ompl.kavrakilab.org/ |
Title | PDT |
Description | Publish open source library for performing and analysing planning experiments |
Type Of Technology | Software |
Year Produced | 2022 |
Open Source License? | Yes |
Impact | Sharing of planning methodology with other research groups and industry to be applied and improved in multiple applications |
URL | https://robotic-esp.com/code/pdt |
Title | TMIT* |
Description | Publish TAMP algorithm open source |
Type Of Technology | Software |
Year Produced | 2022 |
Open Source License? | Yes |
Impact | Diverse use by by other research groups in academia and industry |
URL | https://robotic-esp.com/code/tmitstar/ |
Description | "Lifting The Veil Over Emergency Scene Understanding" - Keynote at the 24th International Conference on Distributed Computng and Networking (ICDCN) 2023. |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Postgraduate students |
Results and Impact | Title "Lifting The Veil Over Emergency Scene Understanding" - a keynote on key technical challenges in applying scene understanding techniques in challenging emergency situations |
Year(s) Of Engagement Activity | 2023 |
Description | "Multi-modal Sensing and Scene Understanding in Emergency Scenarios", Keynote, 24th IEEE Wowmom, 2023. |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Postgraduate students |
Results and Impact | Multi-modal Sensing and Scene Understanding in Emergency Scenarios", Keynote, 24th IEEE Wowmom, 2023. |
Year(s) Of Engagement Activity | 2023 |
Description | "Scene Understanding in Challenging Scenarios Using Multi-Modal Sensing", Keynote at 20th IEEE International Conference on Sensing, Communication and Networking (SECON), 2023. |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Postgraduate students |
Results and Impact | "Scene Understanding in Challenging Scenarios Using Multi-Modal Sensing", Keynote at 20th IEEE International Conference on Sensing, Communication and Networking (SECON), 2023. |
Year(s) Of Engagement Activity | 2023 |
Description | "Trustworthy and Explainable Intelligent Systems for Smart City Services", Panel at IEEE SmartComp 2023. |
Form Of Engagement Activity | A formal working group, expert panel or dialogue |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | The panel brought together people from various emergency response teams and the academic community to discuss how to join forces to respond to emergency events. People were there representing city services linked to transport and healthcare. |
Year(s) Of Engagement Activity | 2023 |
Description | ACE-OPS Intl Collaboration Progress Meeting |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Postgraduate students |
Results and Impact | International Collaboration Progress Meerting Introduction and update from Niki Trigoni from Computer Science (Univ. of Oxford) on the Ace-Ops project. Talk/update by Rowan Border from Oxford Robotics Institute on latest research on multi-motion estimation. Talk by Qian Xie from Computer Science on Thermal based Pedestrian detection. Talk by Vu Tran from Computer Science on Two Sided Data Driven UWB error Mitigation for indoor localisation. Talk by Arif Rahman from the university of Virginia will present on "First responders and CPR". Talk by Lahiru Wijayasingha from the University of Virginia on the use of Augmented reality (smart Glasses) to aid a first responder. Andrew Markham from Computer Science presents plans for Ace-Ops virtual workshop later in May 2022. Final round up and next steps by Niki Trigoni. |
Year(s) Of Engagement Activity | 2022 |
Description | ACE-OPS Partner Mtg |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Postgraduate students |
Results and Impact | Each research group presented their progress and discussed future directions for international collaboration. Several research talks were given by Niki Trigoni, Andrew Markham, Risqi Utama Saputra (Cyber Physical Systems, Univ. of Oxford), Jonathan Gammell and Rowan Border (Oxford Robotics Inst., Univ. of Oxford) and Jack Stankovic (Univ. of Virginia). |
Year(s) Of Engagement Activity | 2021 |
URL | http://aceops.cs.ox.ac.uk/ |
Description | ESA Workshop on AI for Space Operations |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Third sector organisations |
Results and Impact | Workshop discussion |
Year(s) Of Engagement Activity | 2020 |
Description | Invited Talk at Cornell University |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Invited talk to the Department of Computer Science at Cornell University in the US |
Year(s) Of Engagement Activity | 2021 |
Description | Invited Talk at ETH Zurich |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Invited Talk for the Institute for Dynamic Systems and Control, ETH Zurich |
Year(s) Of Engagement Activity | 2022 |
Description | Invited Talk at IEEE/RSJ IROS Workshop |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Invited Talk at IEEE/RSJ IROS Workshop on Evaluating Motion Planning Performance |
Year(s) Of Engagement Activity | 2022 |
Description | Invited Talk at Space @ Oxford Week |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Public/other audiences |
Results and Impact | Invited talk at the Space@Oxford event |
Year(s) Of Engagement Activity | 2021 |
Description | Invited Talk at TU Berlin |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Invited talk to the LIS group at TU Berlin |
Year(s) Of Engagement Activity | 2021 |
URL | http://www.youtube.com/watch?v=cJIP2ad5Hwg |
Description | Invited Talk at TU Berlin |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Invited talk to the LIS group at TU Berlin |
Year(s) Of Engagement Activity | 2021 |
URL | http://www.youtube.com/watch?v=cJIP2ad5Hwg |
Description | Invited Talk at University of British Columbia |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Invited talk to the Department of Mechanical Engineering at the University of British Columbia in Canada |
Year(s) Of Engagement Activity | 2021 |
Description | Invited talk at University of Cambridge, Systems Research Group Seminar |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Postgraduate students |
Results and Impact | Invited talk "Scene understanding in emergency response scenarios" - This talk was intended to share with research community the challenges presented in setting up cyber physical systems in emergency scenarios. Advances in sensing, robotics and machine learning have recently led to scene understanding algorithms that have transformed smart city applications, like transport, healthcare and energy. However this technology has found slower uptake in the challenging world of blue light emergency response. In this talk I will present some of the key challenges faced by intelligent sensor systems in emergency situations including dynamic changes of the environment, lack of preinstalled network or sensor infrastructure, and operation in previously unseen and unpredictable situations. I will then present recent multi-modal sensing approaches to addressing these challenges. |
Year(s) Of Engagement Activity | 2022 |
Description | Invited workshop talk |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Invited talk on my research at workshop |
Year(s) Of Engagement Activity | 2020 |
Description | Invited workshop talk |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Invited talk on my research at workshop |
Year(s) Of Engagement Activity | 2020 |
Description | Joint ACE-OPS workshop - autonomy for emergency response: University of Oxford (UK(), Queensland University of Technology (Australia) and University of Virginia (USA) |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Postgraduate students |
Results and Impact | This workshop included a mix of presentations and discussions on state of the art methods for emergency response, combining approaches to wearable- and robot-based perception and cognitive assistance. These included talks and discussions on visual, visual-inertial and multi-motion odometry, mobile autonomy and cognitive assistance. Further discussion uncovered interesting opportunities for joint collaboration (remote due to covid), which members of the workshops are now pursuing independently. |
Year(s) Of Engagement Activity | 2021 |
Description | Keynote at Conference (SensorNets 2022) |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Other audiences |
Results and Impact | Keynote Title: Scene understanding in emergency applications: challenges and lessons learnt Abstract: Emergency situations present some of the most challenging and unusual scenarios for sensing and scene understanding; yet, it is in these situations, where situation awareness is of paramount importance and where technology is needed the most to help protect human lives. In this talk, I will present some of the key challenges faced by sensing and machine learning algorithms in emergency situations, including lack of pre-installed sensing infrastructure, lack of training data, sensor failure, and limited visibility and connectivity. I will then present recent research directions that we have pursued to address these challenges including multi-modal sensing, cross modality training and human-robot interaction. |
Year(s) Of Engagement Activity | 2022 |
URL | https://www.youtube.com/watch?v=HMOGr9YtA64 |
Description | Organisation Activities for New Workshop (Cyber Physical Systems for Emergency Response - CPS-ER) by Andrew Markham, Niki Trigoni and Jack Stankovic |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Organisation of a new workshop (Cyber Physical Systems for Emergency Response - CPS-ER) by Andrew Markham, Niki Trigoni and Jack Stankovic This is a new workshop colocated with CPS-IOT Week 2022 (https://sites.google.com/view/cps-er/) which was organised by the grant's PIs in order to promote awareness and disseminate research outputs in the area of cyber physical systems for emergency response. Several meetings have taken place to organise this event, review papers and finalise accepted papers, as well as coordinate with the organisers of the CPS-IOT Week. The workshop itself will be held virtually in May 2022. |
Year(s) Of Engagement Activity | 2022 |
Description | Panel discussion at IROS Workshop |
Form Of Engagement Activity | A formal working group, expert panel or dialogue |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Panel discussion at IEEE/RSJ IROS Workshop on Evaluating Motion Planning Performance |
Year(s) Of Engagement Activity | 2022 |
Description | Working collaboration group between University of Oxford CS and Engineering Depts and University of Virginia |
Form Of Engagement Activity | A formal working group, expert panel or dialogue |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Study participants or study members |
Results and Impact | This was a workshop across three research groups, University of Oxford Cyber Physical Systems Group, Oxford Robotics Institute (ORI) and University of Virginia, where representatives of these groups gave short 20 minute talks about ongoing work in the ACE-OPS project. Members of the Cyber Physical Systems Group discussed ongoing work on learning to localise with inertial sensors, selective sensor fusion and sequential invariant domain adaptation. Members of the ORI Group presented ongoing work on best view planning for 3D reconstruction and multimotion visual odometry. Members of the University of Virginia group focused on cognitive assistance for first responders. This was followed by a discussion on the application of the above research directions to emergency response and linkages across the research threads. |
Year(s) Of Engagement Activity | 2020 |
Description | Workshop on Sensing, Estimating and Understanding the Dynamic World |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | ICRA workshop on multimotion estimation |
Year(s) Of Engagement Activity | 2020 |
URL | https://robotics.sydney.edu.au/icra-workshop/ |
Description | Workshop on Sensing, Estimating and Understanding the Dynamic World |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | ICRA workshop on multimotion estimation |
Year(s) Of Engagement Activity | 2020 |
URL | https://robotics.sydney.edu.au/icra-workshop/ |
Description | Workshop organisation: CPS-ER |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Postgraduate students |
Results and Impact | SCOPE Emergency responders operate in dangerous conditions. They need to deploy rapidly to an event, and make decisions under stressful and information-limited conditions, at risk to themselves. Advances in autonomous systems e.g. location, semantics, vital sign monitoring, robotic exploration, high level situational awareness, can help to provide more information, both in real-time, and also for post-event reporting/analysis. This half-day workshop seeks to bring an interested community together to foster interaction, collaboration and participation in this important area. The new workshop (Cyber Physical Systems for Emergency Response - CPS-ER) was organised by PIs Andrew Markham and Niki Trigoni, and external collaborator, Jack Stankovic (Univ. of Virginia). This was a new workshop colocated with CPS-IOT Week 2022 (https://sites.google.com/view/cps-er/) which was organised by the grant's PIs in order to promote awareness and disseminate research outputs in the area of cyber physical systems for emergency response. |
Year(s) Of Engagement Activity | 2022 |
URL | https://sites.google.com/view/cps-er/ |