Active and continual learning strategies for deep learning assisted interactive segmentation of new databases

Aim of the PhD Porject:

Develop interactive deep learning approaches to continually segment databases of images for which no previously annotated training databases exist
Design active learning strategies to retrieve on-the-fly, cases whose manual segmentation will be the most informative for continual learning
Create annotation tools that support accelerated adoption of AI for new applications.
Project Description / Background:

Contemporary progresses in machine learning and artificial intelligence have permitted the development of tools that can assist clinicians in exploiting and quantifying clinical data including images, textual reports and genetic information. State-of-the-art algorithms are becoming mature enough to provide automated analysis when provided with enough high-quality training data and when applied to well-controlled clinical studies and trials [1], [2]. It is clear though that producing manual voxel-accurate medical image segmentation labels is tedious, time-consuming and costly as it usually requires profound radiological expertise. Data annotation is often a rate-limiting factor for the development of application-specific deep learning based image segmentation solutions.

In this project, we will focus on designing machine learning approaches to assist and accelerate the manual segmentation of structures of interest across a database, potentially starting from scratch. Adapting deep learning to support new applications while reducing the burden required to collect and annotate datasets for training purposes remains an active research area [7]. This topic shares challenges with domain adaption [3], for example when trying to limit the amount of new annotations required when new generations of scanners are being rolled out. Naively applying a pre-trained model to an imaging source that may slightly differ from the one used to acquire the training data set on which the model was trained indeed often results in dramatic failures. New annotations are often required to confidently bridge the domain gap and validate the performance of domain adaptation techniques.

In such cases, clinicians are typically left with fully manual or generic interactive methods to delineate structures of interest. Interactive deep learning methodologies are emerging to combine rich prior knowledge embedded in retrospective data from previous patients with as-sparse-as-possible annotations provided by clinicians [4], [5]. Yet, these techniques do currently not continue to learn and improve when being used for new cases. Concurrently, algorithms have been designed to exploit weak labels annotated across a data set to train deep neural networks [6]. Again, these methods require manually segmented ground truth for validation purposes and do not learn for being presented with new cases.

This project will consider the problem of gradually annotating an image segmentation dataset with potentially only very little prior knowledge. This is a timely research question which very few machine learning works have considered so far.

Strains on the healthcare system in the UK create an acute need for finding more effective, efficient, safe, and accurate non-invasive imaging solutions for clinical decision-making, both in terms of diagnosis and prognosis, and to reduce unnecessary treatment procedures and associated costs. Medical imaging is currently undergoing a step-change facilitated through the advent of artificial intelligence (AI) techniques, in particular deep learning and statistical machine learning, the development of targeted molecular imaging probes and novel "push-button" imaging techniques. There is also the availability of low-cost imaging solutions, creating unique opportunities to improve sensitivity and specificity of treatment options leading to better patient outcome, improved clinical workflow and healthcare economics. However, a skills gap exists between these disciplines which this CDT is aiming to fill.

Consistent with our vision for the CDT in Smart Medical Imaging to train the next generation of medical imaging scientists, we will engage with the key beneficiaries of the CDT: (1) PhD students & their supervisors; (2) patient groups & their carers; (3) clinicians & healthcare providers; (4) healthcare industries; and (5) the general public. We have identified the following areas of impact resulting from the operation of the CDT.

- Academic Impact: The proposed multidisciplinary training and skills development are designed to lead to an appreciation of clinical translation of technology and generating pathways to impact in the healthcare system. Impact will be measured in terms of our students' generation of knowledge, such as their research outputs, conference presentations, awards, software, patents, as well as successful career destinations to a wide range of sectors; as well as newly stimulated academic collaborations, and the positive effect these will have on their supervisors, their career progression and added value to their research group, and the universities as a whole in attracting new academic talent at all career levels.

- Economic Impact: Our students will have high employability in a wide range of sectors thanks to their broad interdisciplinary training, transferable skills sets and exposure to industry, international labs, and the hospital environment. Healthcare providers (e.g. the NHS) will gain access to new technologies that are more precise and cost-efficient, reducing patient treatment and monitoring costs. Relevant healthcare industries (from major companies to SMEs) will benefit and ultimately profit from collaborative research with high emphasis on clinical translation and validation, and from a unique cohort of newly skilled and multidisciplinary researchers who value and understand the role of industry in developing and applying novel imaging technologies to the entire patient pathway.

- Societal Impact: Patients and their professional carers will be the ultimate beneficiaries of the new imaging technologies created by our students, and by the emerging cohort of graduated medical imaging scientists and engineers who will have a strong emphasis on patient healthcare. This will have significant societal impact in terms of health and quality of life. Clinicians will benefit from new technologies aimed at enabling more robust, accurate, and precise diagnoses, treatment and follow-up monitoring. The general public will benefit from learning about new, cutting-edge medical imaging technology, and new talent will be drawn into STEM(M) professions as a consequence, further filling the current skills gap between healthcare provision and engineering.

We have developed detailed pathways to impact activities, coordinated by a dedicated Impact & Engagement Manager, that include impact training provision, translational activities with clinicians and patient groups, industry cooperation and entrepreneurship training, international collaboration and networks, and engagement with the General Public.