Reflection Aware Visual Simultaneous Localization and Mapping (RA-vSLAM)

Lead Research Organisation: CARDIFF UNIVERSITY
Department Name: Computer Science

Abstract

The project in computer vision and robotics will research the impact of visual reflections in simultaneous localization and mapping (SLAM).

Reflections are ubiquitous in our environment, and are utilised by our vision system to extend the field of view. An example is the use of rear view mirrors on cars. Visual reflections, although provide only 'subtle' information, bring flexibility to our visual perception of the environment. However, such information has not been utilized in computer vision systems. SLAM is a technique originally developed for robotics applications. Visual SLAM (vSLAM) allows a mobile device to incrementally build a map of its environment and localise itself within that map, using only visual information. Even the state-of-the-art vSLAM solutions are based on the assumption that the scenes captured are real and direct, without consideration of those 'virtual' images resulted from reflections. It is an interesting question whether and how reflections can be used to improve the flexibility and robustness of current vSLAM solutions. The answer to it will find important applications in robot navigation, various autonomous systems, VR/AR, etc.
This project will take a step forward to answer this question by considering planar and mirror reflections in indoor vSLAM, and aims to understand the influence of visual reflections in current vSLAM solutions, and develop a reflection-aware vSLAM solution for more flexible and robust mapping and localisation. To achieve the aim, several key aspects will be researched in the project, including analysis of the robustness of current visual SLAM solutions to mirror reflections, exploration of different approaches to detect and localize mirror planes, and utilise the detected reflections for 3D localization and mapping. Evaluation of the developed algorithms may be carried out in both simulation and in real-world.

Publications

10 25 50

Studentship Projects

Project Reference Relationship Related To Start End Student Name
EP/T517951/1 30/09/2020 29/09/2025
2435656 Studentship EP/T517951/1 30/09/2020 29/09/2024 Peter Herbert
 
Description The first aim of the work funded was to understand the influence of visual reflections in current vSLAM. To achieve this, a dataset was collected and used to evaluate existing vSLAM methods in environments containing mirrors. The results of the analysis has passed peer review, and is due for publication.
Exploitation Route The outcomes from this work will continue to motivate development of robust systems and their constituent components (such as feature extraction) in challenging environments containing mirrors. This will have impacts in academic areas (such as mirror detection and planar symmetry detection tasks in computer vision), as well as for applications such as autonomous navigation, AR/VR and 3D reconstruction.
Sectors Aerospace

Defence and Marine

Digital/Communication/Information Technologies (including Software)

Electronics

 
Title MirrEnv 
Description The MirrEnv dataset comprises 49 RGB-D image sequences taken in environments containing planar mirror reflections. A robotic arm was used to collect group truth localisation positions for each sequence. These sequences can be grouped into 7 different trajectory paths, each of these groups containing sequences with different size mirrors present or covered over. 
Type Of Material Database/Collection of data 
Year Produced 2022 
Provided To Others? No  
Impact This dataset was used to analyse existing visual SLAM methods in mirrored environments and motivate further development of robust visual SLAM systems. this dataset might also be used in the future for developing new methods to tackle these challenging scenarios.