VIPAuto: Robust and Adaptive Visual Perception for Automated Vehicles in Complex Dynamic Scenes

Automated Vehicles (AVs) have great potential in revolutionising the existing transportation system into an intelligent ecosystem that can enhance road safety, service accessibility and environmental sustainability. However, this potential is hampered by the inability of the current learning-based visual perception (VP) system that is trained from limited labelled data and thus fails to understand the complex dynamic driving scene. To deal with this problem, VIPAuto aims to develop a series of ground-breaking technologies for creating a generalized VP system and bridging the gap between the limited training data and the endless variations in the real scene. To this end, two significant challenges will be addressed: 1) boosting the scene understanding accuracy of the VP system under adverse weather conditions and 2) enabling the VP system to recognize and incrementally learn anomalous objects. To tackle the first challenge, a self-supervised domain adaptation strategy will be developed to enable the VP model to learn from unlabelled data by transferring knowledge from the clear weather domain to the adverse weather domain, which is empowered by innovatively established inter- and intra-domain common knowledge. To tackle the second challenge, a few-shot incremental learning strategy will be created to enable the VP model to learn unknown objects by designing contrastive learning to repel unknown objects from known classes and creating an advanced cognitive theory-based representation to promote learning capacity from a few samples. The proposed solutions will be integrated into an optimized VP system and evaluated under the complex dynamic driving scene. VIPAuto will provide theoretical foundations and practical techniques for incrementally adaptive VP technologies, thereby promoting the robustness of scene understanding in the real world to support the decision-making of AVs, and contributing to the EU's long-term goal of "Vision Zero" (zero road fatalities) by 2050.