Accelerating adoption of trustworthy AI in radiology: scalable software for non-technical clinical users to independently validate commercial products at local sites

We propose a novel and radical solution that allows healthcare providers and hospitals to rapidly evaluate and test radiology AI products in an independent manner, at a fraction of the resource cost of current evaluation frameworks.

An increasing and aging population, with more long-term health conditions, is putting increasing strain on the healthcare system. AI can alleviate this if adopted safely and effectively \[1\]. However, current pathways to examine the trustworthiness and performance of commercial AI products are neither standardised nor scalable.

Over 80% of hospital pathways involve imaging and its expert interpretation by radiologists \[11\]. However, England alone had a shortfall of 1,453 clinical radiologist consultants (2021), with clinical radiology directors reporting insufficient numbers for safe and effective patient care. \[11\]. AI and machine learning tools in clinical radiology promise to improve detection rates, streamline clinical workflow and improve patient safety. However, despite early enthusiasm, adoption into routine practice has been slow \[2\].

A key reason for this is that, prior to deployment, each product must be externally and independently validated on local data to ensure reproducibility, generalisability and trustworthiness, along with operational performance and points of failure \[3\]. However, many studies and trials evaluating radiology AI products are hampered by bias, lack of blinding and small datasets and population numbers \[5\]. Furthermore, AI products operate in so-called "black-boxes" whereby the method in which the product operates is unclear to the lay user, creating issues around credibility and trustworthiness.

This poses significant cost and potential clinical risk to institutions looking to deploy AI, creating a major barrier to widespread adoption. There is a clear unmet need to develop less resource intensive processes by which organizations can substantiate medical device manufacturer claims and identify products that offer clear clinical benefit to healthcare providers.

We propose to create a standardised system to validate commercial healthcare imaging AI products at local clinical sites before procurement. Our solution assesses the performance, fairness, robustness and explainability of a black-box product. This can be used by non-technical clinical managers to quickly evaluate and compare AI products, ensuring suitability for their local populations and work schemes.