DEEP LEARNING AND MEDICAL IMAGING

Medical imaging seeks to reveal the structures and activities inside the human body that are normally invisible behind other body parts such as skin and bones. With a wide range of imaging technologies such as radiography, magnetic resonance imaging, ultrasound, echocardiography and tomography, it has long been of great importance in clinical practices for measurement, identification, location and structural analysis of targeted organs, tissues or areas of abnormality. In this project, we will address the following key problems of medical imaging using deep learning methods.

1. Segmentation and structural analysis. Within the commonly used architecture of U-Net, we propose to add the residual blocks to protect the low-level information transmission and recurrent blocks to capture the global spatial structure. However, the end-to-end architectures, even with the directly links from the down-sampling layers to the up-sampling layers, may still suffer from inaccurate segmentation at details. To address this problem, we will work on further refinement of the results using fast graph search methods. Also, to improve the segmentation accuracy at multiple scales, we will concatenate the output from various up-sampling layers to the output.

2. Image registration. The objective is to establish the pixel-to-pixel alignment between a pair of input images from the same patient at different time, from different patients, or images from different imaging modalities. We will work on the conventional feature-based image registration methods as a baseline for this problem. In addition, we propose to develop a deep regression neural network that explicitly use the structural information to guide the search of the deformation mappings. The mappings are modelled into the cost function therefore alignment by these structures is enforced in the learning process.

3. Image generation. Training a model for segmentation of structures or classification of diseases required labelled data, but labelling large set of images is a tedious and time-consuming process. We propose to use synthesised images with their corresponding labels to augment the training data set to improve the performance. We will investigate the potential performance gain from an augmented data set using the Generative Adversarial Network (GAN). Various types of image generation will be investigated, including from images to target structures, from structures to images, and between images of different imaging modalities