SmartHeart: Next-generation cardiovascular healthcare via integrated image acquisition, reconstruction, analysis and learning

Lead Research Organisation: Imperial College London
Department Name: Dept of Computing


The vision for our research programme is to pave the way for a fundamentally different approach in which cardiovascular diseases (CVD) are diagnosed, monitored and treated: We propose to develop a diagnosis-driven "smart Magnetic Resonance (MR) scanner" that it is no longer a mere imaging device but instead becomes a highly sophisticated diagnostic tool. The output of a patient scan with the proposed smart MR scanner will not be just an image, but instead a comprehensive diagnostic assessment and interpretation of the patient's cardiovascular health/disease, enabling optimal treatment decisions for best patient outcome.

The current approach to cardiovascular MR imaging (cMRI) is essentially serial: image acquisition is followed by image analysis and clinical interpretation. In addition, cardiac/respiratory motion is currently resulting in long scanning times for cMRI, with only a small fraction of the data (10-20%) being used for image reconstruction. This leads to breath-holds that are difficult to tolerate by sick patients. Furthermore, the characterization of clinically relevant tissue parameters requires the acquisition of multiple images which is inefficient. The absolute quantification of tissue parameters also remains a major technical challenge, leading to difficulties in interpreting tissue contrast parameters across scanners, clinical centres and patient populations. Finally, the objective interpretation of comprehensive, multi-parametric cMRI in the context of other complex non-imaging data is highly challenging for clinicians.

We propose a transformative approach in which acquisition, analysis and interpretation are tightly coupled, with feedback between the different stages in order to optimize the overall objective: Extracting clinically useful information. Developing such an integrated approach to cardiac imaging will enable rapid, continuous and comprehensive imaging that is both simpler and more efficient than current practice, eliminating "dead time" between separate specialized acquisitions and allowing extraction of multiple dynamic as well as tissue contrast parameters simultaneously.

Planned Impact

Cardiovascular diseases (CVD) cause more than a quarter of all deaths in the UK (155,000 deaths pa). The cost to the UK of premature death, lost productivity, hospital treatment and prescriptions relating to CVD is estimated at £19B each year, with healthcare costs alone totalling an estimated £8B (source: BHF). The current gold standard for diagnosis is based on cardiac catheterisation to obtain coronary angiograms in order to measure parameters such as coronary stenosis and fractional flow reserve (FFR). This highly invasive and costly procedure is increasingly being replaced by imaging modalities such as cardiac Magnetic Resonance Imaging (cMRI). This programme grant will develop smart and integrated image acquisition, analysis and interpretation approaches for cMRI that enable far more accurate, reproducible and objective quantification of CVD. This has the potential to boost clinical decision making in terms of diagnostic and prognostic accuracy. The main beneficiaries are listed below:

The primary beneficiaries of the proposed research are patients with CVD as well as their families. They will benefit from improved diagnosis and prognosis of CVD which is crucial deciding on treatment strategies and preventative strategies for reducing further damage to the heart. This not only has the potential to improve the rehabilitation of patients with CVD but also can improve their quality of life following CVD. Another group of beneficiaries are clinicians (especially cardiologists and radiologists) involved in the care of patients with CVD as well as charities promoting cardiovascular health. They will benefit from better quantitative and objective information for making clinical decisions regarding patient management and as well as from increase confidence via the decision support tools developed in this project. Furthermore, the stratification tools developed here are a prerequisite for developing and evaluating novel therapeutic treatments and are thus of high interest for the pharma and medical device industry.

The healthcare industry also benefits from the research in this project. The medical imaging sector is the biggest component of this industry and will significantly benefit from further developments in medical imaging technologies. This is also illustrated by the fact that two of the world's largest medical imaging companies (Siemens and Philips) are partners in this project. The UK industry is playing a leading role in medical imaging and has a number of rapidly growing SMEs in this area, including Perspectum Diagnostics (project partner), IXICO (co-founded by Rueckert) and Intelligent Ultrasound (co-founded by Noble). In addition to the exploitation of technology and intellectual property, industry will also benefit from a pool of highly-skilled and trained researchers in this area.

Finally, society as a whole will benefit via improved diagnosis of patients as well as better prognosis of outcome and recovery, leading to improved healthcare economics. This will also benefit healthcare providers such as the NHS and policy makers.


10 25 50

publication icon
Aung N (2020) The Effect of Blood Lipids on the Left Ventricle: A Mendelian Randomization Study. in Journal of the American College of Cardiology

publication icon
Bai W (2018) Automated cardiovascular magnetic resonance image analysis with fully convolutional networks. in Journal of cardiovascular magnetic resonance : official journal of the Society for Cardiovascular Magnetic Resonance

Description As part of this grant we have developed (a) novel MR image sequences for cardiac imaging, (b) novel MR image reconstruction techniques and (c) novel MR image analysis approaches. In addition, we are in the process of assembling a large database of clinical cardiac MRI.
Exploitation Route The algorithms and techniques developed so far may be adopted by medical imaging companies and integrated into their products.
Sectors Healthcare

Description Efficient and Robust Assessment of Cardiovascular Disease Using Machine Learning and Ultrasound Imaging
Amount £397,985 (GBP)
Funding ID EP/R005982/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 02/2018 
End 01/2021
Description Healthcare Impact Partnership
Amount £932,050 (GBP)
Funding ID EP/P023509/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 10/2017 
End 09/2020
Description London Medical Imaging & Artificial Intelligence Centre for Value-Based Healthcare
Amount £9,985,272 (GBP)
Funding ID 104691 
Organisation Innovate UK 
Sector Public
Country United Kingdom
Start 02/2019 
End 01/2022
Description MRC Industrial CASE studentship
Amount £104,565 (GBP)
Funding ID MR/N018028/1 
Organisation Medical Research Council (MRC) 
Sector Public
Country United Kingdom
Start 04/2017 
End 03/2021
Description Project grant
Amount £707,983 (GBP)
Funding ID EP/R005516/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 02/2018 
End 01/2021
Description euCanSHare
Amount € 6,039,980 (EUR)
Funding ID 825903 
Organisation European Commission H2020 
Sector Public
Country Belgium
Start 12/2018 
End 11/2022
Description Development of cardiac MRI radiomics as a novel imaging biomarker 
Organisation University of Barcelona
Country Spain 
Sector Academic/University 
PI Contribution Clinical and imaging expertise by Steffen Petersen group.
Collaborator Contribution Machine learning and radiomics expertise by University of Barcelona team.
Impact Multi-disciplinary teams: computer science, data science, cardiology, imaging, genetics. Outcomes: grant submissions, publications as listed elsewhere.
Start Year 2017
Description Heartflow collaboration 
Organisation HeartFlow
Country United States 
Sector Private 
PI Contribution HeartFlow offers a non-invasive method to detect and quantify blockages in arteries by using 3D scans of patients' hearts to simulate blood flow and aid diagnosis.We are assisting the company in developing new algorithms that 'learn' and improve as they analyse datasets to provide increasingly accurate models.
Collaborator Contribution Heartflow provides funding for research on CT image reconstruction and analysis.
Impact New collaboration, no results yet
Start Year 2018
Description NIHR Cardiovascular HTC 
Organisation National Institute for Health Research
Country United Kingdom 
Sector Public 
PI Contribution The NIHR Cardiovascular HTC was a collaborator on the original MedIAN bid. We share information on events between our networks and have run an event together.
Collaborator Contribution We share information on events between our networks and have run an event together.
Impact A joint event was held with follow-up including connecting with the UK Biobank.
Start Year 2014
Description NIHR Cardiovascular MIC 
Organisation National Institute for Health Research
Country United Kingdom 
Sector Public 
PI Contribution The NIHR Cardiovascular MIC replaced the HTC of the same name in January 2018 and we continue to work with them on sharing information and will run a joint meeting in 2018.
Collaborator Contribution See above.
Impact Sharing information and running joint events.
Start Year 2018
Description UK Biobank 
Organisation UK Biobank
Country United Kingdom 
Sector Charity/Non Profit 
PI Contribution The UK Biobank Imaging consortium is gathering a large amount of image data on study subjects. MedIAN participants have discussed with UK Biobank consortium partners challenges in automated image analysis of the cardiac image data.
Collaborator Contribution Knowledge exchange. Some partners have followed up with UK Biobank to consider how they can support analysis. One example is the SmartHeart Programme Grant.
Impact Individuals followed up from the workshop to consider how they might support the UK Biobank.
Start Year 2016
Description International Conference on Image Computing and Computer Assisted Intervention (MICCAI) 2018. Granada, Spain - Program Chair Prof Julia Schnabel 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact This was the largest MICCAI conference to-date, with over 1500 participants from academic and industry.
Year(s) Of Engagement Activity 2018
Description Leaflet about the use of AI in cardiac imaging, in the context of SmartHeart 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact The SmartHeart team have produced a leaflet in the style of a graphic novel to reach out to young audiences, and the public generally, to explain how machine learning is used to aid cardiac imaging, in the context of the SmartHeart programme.
Year(s) Of Engagement Activity 2019,2020
Description Medical Imaging Summer School (MISS 2018): Medical Imaging meets Deep Learning - Director Prof Julia Schnabel 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact The focus of this Medical Imaging Summer School (MISS) is to train a new generation of young scientists to bridge this gap, by providing insights into the various interfaces between medical imaging and deep learning, based on the shared broad categories of medical image computing, computer-aided image interpretation and disease classification. The course will contain a combination of in-depth tutorial-style lectures on fundamental state-of-the-art concepts, followed by accessible yet advanced research lectures using examples and applications. A broad overview of the field will be given, and guided reading groups will complement lectures. The course will be delivered by world renowned experts from both academia and industry, who are working closely at the interface of medical imaging/deep learning.

The school aims to provide a stimulating graduate training opportunity for young researchers and Ph.D. students. The participants will benefit from direct interaction with world leaders in medical image computing and deep learning(often working in both fields). Participants will also have the opportunity to present their own research, and to interact with their scientific peers, in a friendly and constructive setting.
Year(s) Of Engagement Activity 2018
Description One day conference "AI in Cardiac Imaging" MedIAN sponsoring travel grants for early career researchers to attend 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact This conference targeted cardiologists and radiologists using or interested in becoming involved with AI approaches to MRI and PET/MRI, biomedical engineers (working in industry and pre- and post-doctoral students) interested in machine learning, and information and data scientists interested in how AI can be used in diagnosis of cardiovascular diseases. This brought together international leaders and researchers in machine learning, computational imaging, cardiac imaging and cardiology and featured high-profile speakers, posters and demonstrations from industry, academia, and clinical settings which all gave an overview of the clinical challenges, and disseminated the latest advances in cardiac imaging and AI, and how these can be combined to create a personalised diagnostic tool for cardiovascular diseases.
The intended purpose and outcomes/impact were summed up in the blog one of our travel grant recipients described after the event: "The day covered a wide range of different applications for AI within cardiac imaging - some being at a very early stage, e. g. going directly from MRI raw data to diagnosis, whereas others had been used in thousands of patients. Overall, I am happy that much of the earlier scepticism towards the use of machine learning within cardiac imaging seems to be gone (biased sampling I know...), which allows for those fruitful inter-disciplinary conversations that are necessary before a widespread clinical adoption can take place. "

Topics discussed included
AI for cardiac image acquisition and reconstruction
AI for automated cardiac imaging and QC
AI for cardiac image analysis and diagnostics
Extracting quantitative biomarkers using AI
Mining population and cohort studies using AI
Multi-modal (imaging, text, omics) AI approaches
AI-based decision support for CVD
Explainable and interpretable AI
Translation of AI-based solutions into the clinic
Year(s) Of Engagement Activity 2019