MultiHeart: Fusing CT and MRI with Space-Time Transformation Networks

Aim of the PhD Project

The overall goal is to develop a deep-learning based multi-modal spatio-temporal atlas, which will enable rapid co-registration of patient cardiac anatomy between different modalities. Specific aims are:

Automatically register CT and MRI exams within and between patients.
Predict motion abnormalities from CT
Predict extent of ischemic region from CT
Project description

Coronary artery disease is the leading cause of death worldwide. Coronary computed tomography angiography (CCTA) and cardiac magnetic resonance (CMR) are non-invasive imaging methods which are commonly used to evaluate patients to test whether invasive procedures are necessary [1]. However, they provide complimentary information, leading to the need for combining information across modalities for improved diagnosis [2-4]. CCTA provides high-resolution anatomical information allowing for the evaluation of coronary artery stenosis (narrowed arteries), including the characterization of plaque, calcium, percent stenosis [5], and flow limiting stenoses [6]. CMR provides information about myocardial function, including assessment of wall motion abnormalities (from cine-CMR), viability, ischemia and scar (from perfusion-CMR and delayed-enhancement CMR) [1]. The high temporal resolution of cine-CMR allows for wall motion abnormalities to be observed, while the high contrast sensitivity of perfusion-CMR allows for the quantification of myocardial perfusion, as well as identification of microvascular disease [7]. This information complements the assessment of coronary artery disease from CCTA and offers a more complete picture, helping to guide treatment options.

Fusion of CCTA and CMR examinations will give added clinical utility to existing analyses, by providing a deeper and concordant evaluation of anatomical and functional predictors. It would also enable better identification of which stenoses are significant. Perfusion imaging-derived myocardial blood flow could provide improved patient-specific boundary conditions, allowing total flow through the coronary tree to be matched with the measured flow at a whole-heart or territory level. Such co-registration would also allow for validation of a CCTA-derived computation of extent of ischaemic region [8] and allow better parametrization of a CCTA-derived simulated perfusion model [9]. In addition to this application, a joint CT-MRI spatio-temporal motion atlas could be learned. Such a spatio-temporal motion atlas could leverage the high temporal information in cine-MRI to inform the estimation of motion abnormalities from CCTA alone, where only limited temporal resolution (e.g., several frames across the cardiac cycle) might be available.

This PhD project will focus on the development of a deep-learning based multi-modal spatio-temporal atlas, which will also enable rapid co-registration of patient cardiac anatomy between different modalities. This work aims to build on other recent work in this space, such as Atlas-ISTN [10]. An effective multi-modal registration approach would further enable exciting applications of statistical population modelling of anatomy and function, leveraging the complementary information in heterogeneous datasets providing novel insights about normal and pathological variations.

Strains on the healthcare system in the UK create an acute need for finding more effective, efficient, safe, and accurate non-invasive imaging solutions for clinical decision-making, both in terms of diagnosis and prognosis, and to reduce unnecessary treatment procedures and associated costs. Medical imaging is currently undergoing a step-change facilitated through the advent of artificial intelligence (AI) techniques, in particular deep learning and statistical machine learning, the development of targeted molecular imaging probes and novel "push-button" imaging techniques. There is also the availability of low-cost imaging solutions, creating unique opportunities to improve sensitivity and specificity of treatment options leading to better patient outcome, improved clinical workflow and healthcare economics. However, a skills gap exists between these disciplines which this CDT is aiming to fill.

Consistent with our vision for the CDT in Smart Medical Imaging to train the next generation of medical imaging scientists, we will engage with the key beneficiaries of the CDT: (1) PhD students & their supervisors; (2) patient groups & their carers; (3) clinicians & healthcare providers; (4) healthcare industries; and (5) the general public. We have identified the following areas of impact resulting from the operation of the CDT.

- Academic Impact: The proposed multidisciplinary training and skills development are designed to lead to an appreciation of clinical translation of technology and generating pathways to impact in the healthcare system. Impact will be measured in terms of our students' generation of knowledge, such as their research outputs, conference presentations, awards, software, patents, as well as successful career destinations to a wide range of sectors; as well as newly stimulated academic collaborations, and the positive effect these will have on their supervisors, their career progression and added value to their research group, and the universities as a whole in attracting new academic talent at all career levels.

- Economic Impact: Our students will have high employability in a wide range of sectors thanks to their broad interdisciplinary training, transferable skills sets and exposure to industry, international labs, and the hospital environment. Healthcare providers (e.g. the NHS) will gain access to new technologies that are more precise and cost-efficient, reducing patient treatment and monitoring costs. Relevant healthcare industries (from major companies to SMEs) will benefit and ultimately profit from collaborative research with high emphasis on clinical translation and validation, and from a unique cohort of newly skilled and multidisciplinary researchers who value and understand the role of industry in developing and applying novel imaging technologies to the entire patient pathway.

- Societal Impact: Patients and their professional carers will be the ultimate beneficiaries of the new imaging technologies created by our students, and by the emerging cohort of graduated medical imaging scientists and engineers who will have a strong emphasis on patient healthcare. This will have significant societal impact in terms of health and quality of life. Clinicians will benefit from new technologies aimed at enabling more robust, accurate, and precise diagnoses, treatment and follow-up monitoring. The general public will benefit from learning about new, cutting-edge medical imaging technology, and new talent will be drawn into STEM(M) professions as a consequence, further filling the current skills gap between healthcare provision and engineering.

We have developed detailed pathways to impact activities, coordinated by a dedicated Impact & Engagement Manager, that include impact training provision, translational activities with clinicians and patient groups, industry cooperation and entrepreneurship training, international collaboration and networks, and engagement with the General Public.