Reflection Aware Visual Simultaneous Localization and Mapping (RA-vSLAM)

The project in computer vision and robotics will research the impact of visual reflections in simultaneous localization and mapping (SLAM).

Reflections are ubiquitous in our environment, and are utilised by our vision system to extend the field of view. An example is the use of rear view mirrors on cars. Visual reflections, although provide only 'subtle' information, bring flexibility to our visual perception of the environment. However, such information has not been utilized in computer vision systems. SLAM is a technique originally developed for robotics applications. Visual SLAM (vSLAM) allows a mobile device to incrementally build a map of its environment and localise itself within that map, using only visual information. Even the state-of-the-art vSLAM solutions are based on the assumption that the scenes captured are real and direct, without consideration of those 'virtual' images resulted from reflections. It is an interesting question whether and how reflections can be used to improve the flexibility and robustness of current vSLAM solutions. The answer to it will find important applications in robot navigation, various autonomous systems, VR/AR, etc.
This project will take a step forward to answer this question by considering planar and mirror reflections in indoor vSLAM, and aims to understand the influence of visual reflections in current vSLAM solutions, and develop a reflection-aware vSLAM solution for more flexible and robust mapping and localisation. To achieve the aim, several key aspects will be researched in the project, including analysis of the robustness of current visual SLAM solutions to mirror reflections, exploration of different approaches to detect and localize mirror planes, and utilise the detected reflections for 3D localization and mapping. Evaluation of the developed algorithms may be carried out in both simulation and in real-world.