Self-supervised Monocular Depth Estimation

"1.1 Monocular Depth Estimation
A fundamental task in computer vision is understanding our and other objects'
3D positioning in space. Many prior methods of depth estimation made use of stereoscopic depth, while this was an effective technique, recent advancements in Deep Neural Networks have led to the development of monocular depth estimation. Monocular depth estimation hallucinates depth from a single RGB image. Traditionally, supervised methods would have been used to determine the depth of an image per pixel as a regression task. However, recent developments use self-supervising methods that take advantage of view-synthesis between consecutive RGB frames. These methods make use of ego-pose estimation, and depth estimation to inverse warp an image frame to its consecutive image frame. This leads to the ability to not only understand the depth of a scene but to understand the camera's
motion through this scene. These methods have shown promising results and are constantly improving but there are many advancements to be made.
1.2 Improvements
It is now a well-known issue that these methods struggle with dynamic objects, as they assume rigid motion in all scenes. Most recent methods attempt to avoid these downfalls by allowing the system to ignore the regions that cause the error in the loss function. In this project, we aim to employ these inconveniences to teach the depth and ego-pose networks more about the scene. Also, by focusing on these dynamic objects, we can track the motion
of vehicles and pedestrians in a self-supervised manner, which is a vital task for autonomous driving. Furthermore, there has been a large focus on scenes that only take into consideration daylight videos with mostly sunny clear weather. Taking today's SoTA models we see that they struggle with rain, snow, fog and other more extreme weather events leading to failure cases whenever the weather is not clear. Given that in 2021 we had 149 days of rain in the UK it is a clear issue for our depth and pose models to handle rain. Modern methods attempt to handle this by fine-tuning the models for different weather events, leading to separate models for each weather condition, which demonstrates that the current depth models do not generalise well. We aim to improve this methodology while leading to much greater generalisability of the networks. As these methods are ill-posed, it is beneficial for these methods to have some form of uncertainty estimations for the depth and pose networks. Current work in this area is limited, and uncertainty estimation work needs to be developed for ego-pose and other object-pose estimations. This and further improvements for depth uncertainty would have direct industrial applications and would be a focus of this project.
Penultimately, these methods struggle with texture-less regions, which has been addressed in previous work. To further these methods, we will use a more efficient loss function for view synthesis in textureless regions. Finally, we aim to handle all of these issues while leading to reductions in computation usage of the models, allowing for these methods to be applied to practical applications.
1.3 Industry Application
Most self-driving vehicles use a sensor fusion setup of Radar, Li-DAR and stereo sensors. This setup generates accurate 3D reconstructions but leads to significant physical costs. This combination of sensors can suffer in inaccurate results as sensors may disagree. Ultimately, replacing this configuration with a simple, accurate, monocular camera setup around the vehicle would reduce the need for Li-DAR and stereo cameras for each vehicle in a fleet. While this improvement is moderate for a single vehicle, on a large fleet this would lead to compelling reductions in costs.
"