Improving the Transparency of Machine Learning and Intelligent Systems for Autonomy

Many recent developments in machine intelligence focus on the quality of training and datasets that are fed into the learning algorithms. The shortfalls of an intelligent system can often be attributed to flaws in the training data, however it is not practical to provide an exhaustive training dataset to guarantee robust behaviour in extreme operating scenarios. On the other hand, the inner workings of intelligent algorithms, including learning algorithms, are often treated as a "black box". This approach - combined with reliance on the quality of training data - poses a huge challenge for the validation and interpretability of AI systems as a whole. This project will develop methods for inspecting the inner workings of representative AI algorithms and methodologies for increasing robustness.

This project aims to create tools and methods for interpretability and transparency, with a focus on applications for transport systems, e.g., to support safety assurance and autonomy. The project also aims to develop tools and models to draw out "what has been learnt and why" by an AI system. The objectives of the project are (i) to understand which of the established and recent machine learning (ML) techniques are most relevant to current and anticipated intelligent transport systems, (ii) understand the mapping between these ML techniques, the applications, and existing approaches to transparency such as proxy models and salience maps, and identify the primary challenges, and (iii) develop new methods for transparency to address some of these challenges. The main novelty of the research comes from understanding how transparency impacts on ML applications of intelligent transport applications, and in the creation of methods to address some of the limitations of existing techniques.

This project is in collaboration with TRL Limited who in addition to partly funding the project will contribute staff time to supervise the student and access to proprietary data with corresponding support.