10065751Determining Real World AI Trustworthiness and RobustnessClosedCollaborative R&DInnovate UKRecent advances in Machine Learning have dramatically expanded tasks that could be automated. offering enormous opportunities for improving public services and boosting economic growth. One area where advances have been largest is Computer Vision. However, deploying Computer Vision systems - particularly in safety critical environments -- is problematic. Systems with machine learning tend to fail in unexpected ways that lead to substandard performance when deployed, but also reputational or regulatory damage to the organization making use of the technology. These failures of machine learning systems can either be due to engineering or societal expectations. Engineering failures are problems with data collection and model training, such as imbalanced data, model drift, poor out-of-distribution detection, adversarial attacks etc. A system may fail to meet societal expectations when it exhibits behaviours that are unethical; for example, poor predictions or decisions related to protected characteristics such as race or gender. The propensity of these failure modes should be measured and mitigated for an AI to be trustworthy. In this project, Advai will create sandbox test environments for Computer Vision systems and metrics for reliably predicting these failure modes. The sandbox environments will be independent of the model development process and act as proxy for real-world deployment without assuming the risk of genuine failure post-deployment. An efficient pipeline for creating third party testing environments would provide two benefits for Accelerating Trustworthy AI: 1.Providing an indication of real-world performance during development. This would greatly increase efficiency, as it becomes possible to identify which models will succeed or fail when deployed; 2.Accelerating data production that will, over time, establish which tools are indicative of real-world model failures in deployment. These metrics themselves can be used as indicators of real-world model performance.