Deep Learning to find signal abnormalities in medical images
Lead Research Organisation:
University College London
Department Name: Medical Physics and Biomedical Eng
Abstract
Using advanced machine learning techniques, namely deep learning, we aim to automatically detect problematic medical images containing imaging artefacts. For example, in Magnetic Resonance Imaging (MRI) these artefacts may include motion artefacts (caused by patient movement during scanning), poor acquisitions (e.g. wrong sequence or parameters), and mislabelling (e.g. scanned the wrong body part). Furthermore, we aim to make commonplace deep learning tasks, such as MR image segmentation (extracting part of the image), robust to the presence of such artefacts. This analysis will not only enable a real time data-quality warning system for clinicians and radiographers, but also ensure that input MR image data is correct and of sufficient quality for fully-automatic image analysis.
To answer these questions I will be taking the following approach:
1. Exploring the different kinds of image artefacts that can occur in MRI and understanding the underlying physics that causes such artefacts.
2. Developing mathematical models of the processes that cause these image artefacts and implementing them in software to synthesise realistic MRI artefacts. For example, by modelling a patient's head movement during a scan we can generate new images which contain motion artefacts.
3. Training deep learning models by augmenting the training data using the proposed artefact synthesis model. The artefact model can be included in a neural network as an augmentation layer which randomly generates artefacts during training, effectively increasing the amount and variability of the data seen by the network.
4. Evaluating the performance of deep learning models trained using the artefact model on real-world MR images which contain image artefacts. The aim is that the models trained using synthesised artefacts will perform better (i.e. produce more accurate segmentations) than models that haven't been trained with artefacts.
5. Use the artefact model to estimate a measure of uncertainty in the predicted segmentations, i.e. if an input image contains an artefact we would expect the uncertainty in the output segmentation to be higher.
6. Attempt to classify the presence of an artefact in real-world images, i.e. automatically detect if an image contains an artefact or not, and possibly even locate which parts of the image contain the artefact.
The novel engineering aspect of this project is to be able to augment training data for deep learning using synthesised MRI artefacts. In theory, this will allow us to generate infinite amounts of training data as artefacts can be randomly generated on-the-fly during training. In deep learning it is common to augment training data by simply rotating, cropping, and flipping input images, but we hope to show that our proposed artefact augmentation model will more accurately perform tasks such as segmentation on real-world data containing artefacts compared to the more "classical" augmentation techniques.
Previous work in MRI artefacts has mainly focused on designing ways to correct for them (i.e. 'un-doing' the artefact). While it may be possible to learn to correct for artefacts and then perform a deep learning task such as segmentation, we argue, as it is common in deep learning, that one should optimise for the final task in an end-to-end manner. Furthermore, it is likely that certain types of artefacts will not to be removable, and thus cause problems when running a segmentation model on partially-corrected data. We do expect a minor drop in segmentation accuracy when an artefact augmented model is applied to clean data, but we aim for this drop to be not statistically significant, but provided a significant improvement on artefact data. While overall model performance is important and should be a key goal, robustness to data artefacts is paramount to enable the safe clinical translation of such techniques.
To answer these questions I will be taking the following approach:
1. Exploring the different kinds of image artefacts that can occur in MRI and understanding the underlying physics that causes such artefacts.
2. Developing mathematical models of the processes that cause these image artefacts and implementing them in software to synthesise realistic MRI artefacts. For example, by modelling a patient's head movement during a scan we can generate new images which contain motion artefacts.
3. Training deep learning models by augmenting the training data using the proposed artefact synthesis model. The artefact model can be included in a neural network as an augmentation layer which randomly generates artefacts during training, effectively increasing the amount and variability of the data seen by the network.
4. Evaluating the performance of deep learning models trained using the artefact model on real-world MR images which contain image artefacts. The aim is that the models trained using synthesised artefacts will perform better (i.e. produce more accurate segmentations) than models that haven't been trained with artefacts.
5. Use the artefact model to estimate a measure of uncertainty in the predicted segmentations, i.e. if an input image contains an artefact we would expect the uncertainty in the output segmentation to be higher.
6. Attempt to classify the presence of an artefact in real-world images, i.e. automatically detect if an image contains an artefact or not, and possibly even locate which parts of the image contain the artefact.
The novel engineering aspect of this project is to be able to augment training data for deep learning using synthesised MRI artefacts. In theory, this will allow us to generate infinite amounts of training data as artefacts can be randomly generated on-the-fly during training. In deep learning it is common to augment training data by simply rotating, cropping, and flipping input images, but we hope to show that our proposed artefact augmentation model will more accurately perform tasks such as segmentation on real-world data containing artefacts compared to the more "classical" augmentation techniques.
Previous work in MRI artefacts has mainly focused on designing ways to correct for them (i.e. 'un-doing' the artefact). While it may be possible to learn to correct for artefacts and then perform a deep learning task such as segmentation, we argue, as it is common in deep learning, that one should optimise for the final task in an end-to-end manner. Furthermore, it is likely that certain types of artefacts will not to be removable, and thus cause problems when running a segmentation model on partially-corrected data. We do expect a minor drop in segmentation accuracy when an artefact augmented model is applied to clean data, but we aim for this drop to be not statistically significant, but provided a significant improvement on artefact data. While overall model performance is important and should be a key goal, robustness to data artefacts is paramount to enable the safe clinical translation of such techniques.
Studentship Projects
| Project Reference | Relationship | Related To | Start | End | Student Name |
|---|---|---|---|---|---|
| EP/N509577/1 | 01/10/2016 | 24/03/2022 | |||
| 1939674 | Studentship | EP/N509577/1 | 25/09/2017 | 25/12/2020 | Richard Shaw |
| Description | This work so far has focused on two main areas: 1) simulating motion artefacts in MRI and 2) automated quality control of MRI scans. 1) Patient movement during the acquisition of magnetic resonance images (MRI) can cause unwanted image artefacts. These artefacts may affect the quality of clinical diagnosis and cause errors in automated image analysis. In this work, we have presented a method for generating realistic motion artefacts from artefact-free MRI data to be used in deep learning frameworks, increasing training appearance variability and ultimately making machine learning algorithms such as convolutional neural networks (CNNs) more robust to the presence of motion artefacts. By modelling patient movement as a sequence of randomly-generated rigid 3D transformations, we can generate motion synthetic artefact data. We show that by augmenting the training of segmentation CNNs with these artefacts, we can train models that generalise better and perform more reliably in the presence of artefact data, with negligible cost to their performance on clean data. We have shown that the performance of models trained using artefact data on segmentation tasks on real-world test-retest image pairs is more robust. We also demonstrated that our augmentation model can be used to learn to retrospectively remove certain types of motion artefacts from real MRI scans. Finally, we have shown that measures of uncertainty obtained from motion augmented CNN models reflect the presence of artefacts and can thus provide relevant information to ensure the safe usage of deep learning extracted biomarkers in a clinical pipeline. 2) Quality control (QC) of medical images is essential to ensure that downstream analysis such as segmentation can be performed successfully. Currently, QC is predominantly performed visually at significant time and operator cost. We have aimed to automate the process by formulating a probabilistic neural network that estimates uncertainty, hence providing a proxy measure of image quality that is learnt directly from the data. By augmenting the training data with different types of simulated artefacts, we have designed a novel neural network architecture based on a student-teacher framework to decouple sources of uncertainty related to different kinds of artefacts. This enables us to predict separate uncertainty quantities for the different types of data degradation. While the uncertainty measures reflect the presence and severity of image artefacts, the network also provides segmentation predictions that are as good as possible given the quality of the data. We have shown that models trained with simulated artefacts provide informative measures of uncertainty on real-world images and we validated our uncertainty predictions on problematic images identified by human-raters. |
| Exploitation Route | This work will hopefully enable safer deep learning systems for medical image analysis. |
| Sectors | Digital/Communication/Information Technologies (including Software),Healthcare,Pharmaceuticals and Medical Biotechnology |