Telerobotics Simulation in Mixed Reality

The convergence of state-of-the-art research in spatial computing, machine learning, and teleoperation proposes to transform field, industrial, and medical robotics. The potential positive impact is most observed in scenarios too dangerous to humans, where robotic avatars are irreplaceable. One such scenario is the deep sea. In this report we review literature in several topics surrounding unmanned underwater vehicles (UUVs), teleoperation, and computer vision to identify research gaps. Several gaps are considered, and a mixed reality simulated teleoperation system is proposed. Preliminary computer vision and stereo vision studies are investigated before a synchronised stereo vision system is described for performing depth estimation, object detection and segmentation, and mixed reality experiments in an underwater environment. Future work on variable lighting and turbidity tests in the tank, and context-awareness features of the mixed reality audiovisual aids are also discussed.
RESEARCH AIMS
1. Explore state of the art and challenges in remote-control of unmanned [underwater] vehicles.
2. Develop a computer vision and/or image processing technique for enhancing efficiency and perception of an underwater vehicle and its manipulators during operation.
3. Data-centric simulation modelling of an underwater vehicle in its environment for efficient mixed reality feedback of robot and scene data.
OBJECTIVES
1.
a. Engage with industry stakeholders to collect feedback from human operators of underwater vehicles.
b. Perform an academic literature survey to understand the state of the art and current challenges in mobile robot teleoperation.
c. Explore underwater vehicles market to understand control and sensing capabilities.
2.
a. Perform academic literature survey for computer vision and underwater image processing for underwater vehicles for different applied use cases/scenarios.
b. Explore existing computer vision techniques and frameworks.
c. Implement pipeline for underwater vehicle scenario(s).
d. Test performance of the developed technique for given scenario(s).
e. Develop visualisation technique to support human operator.
3.
a. Perform academic literature survey for data-centric modelling and simulation of remote robots and environments.
b. Further objectives of research aim 3 will be identified at a later stage and will depend on the outcomes of research aims 1 and 2.