Reflection Aware Visual Simultaneous Localization and Mapping (RA-vSLAM)
Lead Research Organisation:
CARDIFF UNIVERSITY
Department Name: Computer Science
Abstract
The project in computer vision and robotics will research the impact of visual reflections in simultaneous localization and mapping (SLAM).
Reflections are ubiquitous in our environment, and are utilised by our vision system to extend the field of view. An example is the use of rear view mirrors on cars. Visual reflections, although provide only 'subtle' information, bring flexibility to our visual perception of the environment. However, such information has not been utilized in computer vision systems. SLAM is a technique originally developed for robotics applications. Visual SLAM (vSLAM) allows a mobile device to incrementally build a map of its environment and localise itself within that map, using only visual information. Even the state-of-the-art vSLAM solutions are based on the assumption that the scenes captured are real and direct, without consideration of those 'virtual' images resulted from reflections. It is an interesting question whether and how reflections can be used to improve the flexibility and robustness of current vSLAM solutions. The answer to it will find important applications in robot navigation, various autonomous systems, VR/AR, etc.
This project will take a step forward to answer this question by considering planar and mirror reflections in indoor vSLAM, and aims to understand the influence of visual reflections in current vSLAM solutions, and develop a reflection-aware vSLAM solution for more flexible and robust mapping and localisation. To achieve the aim, several key aspects will be researched in the project, including analysis of the robustness of current visual SLAM solutions to mirror reflections, exploration of different approaches to detect and localize mirror planes, and utilise the detected reflections for 3D localization and mapping. Evaluation of the developed algorithms may be carried out in both simulation and in real-world.
Reflections are ubiquitous in our environment, and are utilised by our vision system to extend the field of view. An example is the use of rear view mirrors on cars. Visual reflections, although provide only 'subtle' information, bring flexibility to our visual perception of the environment. However, such information has not been utilized in computer vision systems. SLAM is a technique originally developed for robotics applications. Visual SLAM (vSLAM) allows a mobile device to incrementally build a map of its environment and localise itself within that map, using only visual information. Even the state-of-the-art vSLAM solutions are based on the assumption that the scenes captured are real and direct, without consideration of those 'virtual' images resulted from reflections. It is an interesting question whether and how reflections can be used to improve the flexibility and robustness of current vSLAM solutions. The answer to it will find important applications in robot navigation, various autonomous systems, VR/AR, etc.
This project will take a step forward to answer this question by considering planar and mirror reflections in indoor vSLAM, and aims to understand the influence of visual reflections in current vSLAM solutions, and develop a reflection-aware vSLAM solution for more flexible and robust mapping and localisation. To achieve the aim, several key aspects will be researched in the project, including analysis of the robustness of current visual SLAM solutions to mirror reflections, exploration of different approaches to detect and localize mirror planes, and utilise the detected reflections for 3D localization and mapping. Evaluation of the developed algorithms may be carried out in both simulation and in real-world.
Organisations
Studentship Projects
Project Reference | Relationship | Related To | Start | End | Student Name |
---|---|---|---|---|---|
EP/T517951/1 | 30/09/2020 | 29/09/2025 | |||
2435656 | Studentship | EP/T517951/1 | 30/09/2020 | 29/09/2024 | Peter Herbert |
Description | The first aim of the work funded was to understand the influence of visual reflections in current vSLAM. To achieve this, a dataset was collected and used to evaluate existing vSLAM methods in environments containing mirrors. The results of the analysis has passed peer review, and is due for publication. |
Exploitation Route | The outcomes from this work will continue to motivate development of robust systems and their constituent components (such as feature extraction) in challenging environments containing mirrors. This will have impacts in academic areas (such as mirror detection and planar symmetry detection tasks in computer vision), as well as for applications such as autonomous navigation, AR/VR and 3D reconstruction. |
Sectors | Aerospace Defence and Marine Digital/Communication/Information Technologies (including Software) Electronics |
Title | MirrEnv |
Description | The MirrEnv dataset comprises 49 RGB-D image sequences taken in environments containing planar mirror reflections. A robotic arm was used to collect group truth localisation positions for each sequence. These sequences can be grouped into 7 different trajectory paths, each of these groups containing sequences with different size mirrors present or covered over. |
Type Of Material | Database/Collection of data |
Year Produced | 2022 |
Provided To Others? | No |
Impact | This dataset was used to analyse existing visual SLAM methods in mirrored environments and motivate further development of robust visual SLAM systems. this dataset might also be used in the future for developing new methods to tackle these challenging scenarios. |