Intelligent Imaging: Motion, Form and Function Across Scale

Lead Research Organisation: University College London
Department Name: Medical Physics and Biomedical Eng

Abstract

This programme aims to change the way medical imaging is currently used in applications where quantitative assessment of disease progression or guidance of treatment is required. Imaging technology traditionally sees the reconstructed image as the end goal, but in reality it is a stepping stone to evaluate some aspect of the state of the patient, which we term the target, e.g. the presence, location, extent and characteristics of a particular disease, function of the heart, response to treatment etc. The image is merely an intermediate visualization, for subsequent interpretation and processing either by the human expert or computer based analysis. Our objectives are to extract information which can be used to inform diagnosis and guide therapy directly from the measurements of the imaging device. We propose a new paradigm whereby the extraction of clinically-relevant information drives the entire imaging process. All medical imaging devices measure some physical attribute of the patient's body, such as the X-ray attenuation in CT, changes acoustic impedance in ultrasound, or the mobility of protons in MRI. These physical attributes may be modulated by changes in structure or metabolic function. Medical images from devices such as MR and CT scanners often take 10s of seconds to many minutes to acquire. The unborn child, the very young, the very old or very ill cannot stay still for this time and methods of addressing motion are inefficient and cannot be applied to all types of imaging. Usually triggering and gating strategies are applied, which result in a low acquisition efficiency (since most of the data is rejected) and often fail due to irregular motion. As a result the images are corrupted by significant motion artifact or blurring.Accurate computational modeling of physiology and pathological processes at different spatial scales has shown how careful measurements from imaging devices might allow the clinician or the medical scientist to infer what is happening in health, in specific diseases and during therapy. Unfortunately, making these accurate measurements is very difficult due to the movement artifacts described above. Imaging systems can provide the therapist, interventionist or surgeon with a 3D navigational map showing where therapy should be delivered and measuring how effective it is. Unfortunately image guided interventions in the moving and deforming tissues of the chest and abdomen is very difficult as the images are often corrupted by motion and as the procedure progresses the images generally diverge from the local anatomy that the interventionist or surgeon is treating.Our programme brings together three different groups of people: computer scientists who construct computer models of anatomy, physiology, pharmacological processes and the dynamics of tissue motion; imaging scientists who develop new ways to reconstruct images of the human body; and clinicians working to provide better treatment for their patients. With these three groups working together we will devise new ways to correct for motion artifact, to produce images of optimal quality that are related directly to clinically relevant measures of tissue composition, microscopic structure and metabolism. We will apply these methods to improve understanding of disease progression; guide therapies and assess response to treatment in cancer arising in the lung and liver; to ischaemic heart disease; to the clinical management of the foetus while still in the womb; and to caring for premature babies and young children.

Planned Impact

Please see separate 2 page attachment

Publications

10 25 50
publication icon
Shi W (2013) Temporal sparse free-form deformations. in Medical image analysis

publication icon
Mendelson AF (2014) The empirical variance estimator for computer aided diagnosis: lessons for algorithm validation. in Medical image computing and computer-assisted intervention : MICCAI ... International Conference on Medical Image Computing and Computer-Assisted Intervention

publication icon
Vos SB (2017) The importance of correcting for signal drift in diffusion MRI. in Magnetic resonance in medicine

publication icon
Clarkson MJ (2015) The NifTK software platform for image-guided interventions: platform overview and NiftyLink messaging. in International journal of computer assisted radiology and surgery

 
Description The key findings had been entered in previous returns but seem to have disappeared from teh system. The following is an extract of our final report:
Summary of Progress
We have made significant progress in developing new methods for
compensating for respiratory motion, cardiac motion and fetal motion
to improve image quality and hence diagnosis, fetal interventions
and guide radiotherapy in the lung. Our new methods exploit sparsity
and redundancy to speed up image acquisition and reconstruction.
We have devised new ways to compensate for motion and provide
fast reconstruction of MR images of the adult heart and fetus based
on manifold learning and super-resolution using dictionary learning.
Manifold learning has also been used learn respiratory motion. We have
developed a generalised framework that combines motion model fitting
with motion compensated image reconstruction. Our methods have been
applied to MR imaging and ultrasound imaging in the heart and fetus
and CT imaging in the lungs as well as respiratory compensation in PET
imaging. An exciting new innovation is the application of related methods
to a novel photoacoustic imaging device, potentially for assessing
tissue microstructure and vascularity for minimally invasive surgery.
While measuring and compensating for motion is a core theme of the
programme it is important that the effects of motion are minimised as
much as possible at acquisition. We have adapted an MR compatible
Active Breathing Coordinator (ABC) originally developed for radiotherapy
to minimise patient breathing motion during MR acquisition.
Our pioneering work on microstructure imaging, VERDICT, coupled with
image guided biopsy and focal ablation is looking to drive radical changes
in the detection, stratification and management of prostate cancer.
Related tissue microstructure work is shedding light on how the fetus
develops and how we might improve detection of fetal abnormalities and
interventions to mitigate them. The work on fetal and neonatal imaging
is providing new insight into how the normal brain develops and how
this normal development might be disrupted, for example by oxygen
starvation.

Key impacts
We are proud of the impact that our Programme has had in medical
imaging science and clinical translation. Overall the Programme and
related work has resulted in 158 papers in peer reviewed journals up until
April 2015. We have published in the most prestigious technical journals
in our area such as IEEE Transactions on Medical Imaging and Medical
Image Analysis as well as high ranking medical Journals such as Cancer
Research. We have also published a review of respiratory motion models
[McClelland et al 2013].
The following have been awarded prizes at conferences:
• Muhammad Usman: Summa Cum Laude ISMRM 2012
• Shaihan Malik: ISMRM II Rabi Young Investigator Award 2013
• Nikos Dikaios: merit award Summa Cum Laude ISMRM 2013
• Christian Baumgartner: best poster award at IPMI 2013
• Devis Peressutti: best student paper award at MIUA 2013
• Ferrazzi G: Magna Cum Laude ESMRMB 2014
The following PIs have been recognised via invitation to chair or co-chair
the following international conferences:
• ISBI 2012 Co-Chair Daniel Rueckert
• WBIR 2012 Co-chair Daniel Rueckert
• FIMH 2013 Co-Chairs Seb Ourselin, Daniel Rueckert and Nic Smith
• WBIR 2014 co-Chair Seb Ourselin
• IPMI 2015 General Chair Seb Ourselin, co-chair Danny Alexander
• MICCAI 2016 General Chair Seb Ourselin
• ISBI 2015 Programme Chair Seb Ourselin

Dave Hawkes and Martin Leach have both been elected NIHR Senior
Investigators (Martin Leach for a second term).

Sustainability
The Investigators in the Programme have been very successful at raising
funds to sustain the critical mass working in medical imaging sciences
research. The future of imaging research in London and the South
East of the UK is robust and London is now recognised internationally
as a hub of internationally leading imaging research comparable with
Boston, the Bay Area or Toronto. The table below lists the grants that
we have obtained over the last 3 years. Of significance is the award of
a Platform Grant to Alexander et al at UCL and 2 EPSRC Centre for
Doctoral training in Medical Imaging, one based at UCL and one joint
between KCL and Imperial. With a combined intake of between 30 and
40 students per annum this will transform the available expertise in this
area in the UK. The Platform Grant provides core funding to develop new
directions including combining modelling and imaging including sparse
data, dealing with very large and diverse data sources and driving these
through to new clinical areas with the software engineering to support
this. Both CDTs reflect the different research and training excellence of
the three institutions but both provide excellent opportunities for future
collaborations, provide close links with industry but above all will provide
the trained scientists who will grow and transform our discipline in the
future, contributing to both health and wealth in the UK.
CRUK and EPSRC have renewed the funding for the ICR Cancer
Imaging Centres and the joint UCL/KCL Cancer Imaging centre. Also
significant is the buy-in of the NIHR Biomedical research centres at KCL
and UCL. This will ensure investment in the clinical translation of the
new methods we have invented and the new technologies that we have
been developing. Also of note is the 3 major grants in imaging in fetal
medicine from the Wellcome Trust, EPSRC and ERC. Fetal medicine has
emerged as a major application area for the programme, in which the UK
is world leading. In addition we have been awarded significant EPSRC
and EU grants from work started within the programme including work on
combining compressed sensing methods into a novel device for dynamic
photoacoustic imaging, development of endoscopic photoacoustics,
the analysis of tissue microstructure as an indicator of malignancy in
primary breast cancer and surrounding stroma (EU PRISM and EPSRC
MIMIC) and the developments in tissue modelling combined with imaging
for planning and assessing cosmesis in breast cancer surgery (EU
PICTURE). UCL was awarded an MRC capital infrastructure bid "Centre
for image-guided therapy - a theranostic approach to patients with
cancer" covering investments of £5.3M in Hyperpolarised MRI, MR guided
HIFU, Molecular Pathology and Magnetic Particle Therapy. UCL also
led the successful bid to MRC in medical bioinformatics "eMedLab" for
£6.8M for compute infrastructure. The consortium includes 7 institutions,
including KCL, and the Programme Grant fed in requirements for the
imaging component. Programme Grant PIs are also engaged in the
NIHR-HTC Medical Image Analysis Network (MedIAN) led from Oxford in
Cardiovascular and Colorectal Imaging.
Exploitation Route See above
Sectors Digital/Communication/Information Technologies (including Software),Healthcare,Pharmaceuticals and Medical Biotechnology

URL https://www.ucl.ac.uk/medical-image-computing
 
Description As stated in the section on academic impact this programme grant had a major impact on research capacity and quality in the area of medical image computing and image guided interventions. This has ensured that the UK and in particular activity in Imperial, Kings College London and University College London is clearly recognised as world leading. The output of the programme has directly and indirectly led to changes in the way that prostate cancer, lung cancer, breast cancer, heart disease and fetal health have been managed, including changes to NICE guidelines. Work undertaken during the programme has contributed to a number of successful spin-outs, growth of existing companies and joint developments with industry. Participating laboratories have made significant contributions to public and patient education.
First Year Of Impact 2016
Sector Digital/Communication/Information Technologies (including Software),Healthcare,Manufacturing, including Industrial Biotechology
Impact Types Societal,Economic

 
Description impact of prostate MRI developments on guidelines
Geographic Reach Multiple continents/international 
Policy Influence Type Contribution to a national consultation/review
Impact The PROMIS trial and other trials of imaging and assessment of focal therapies on the prostate are influencing national and international guidelines on the use of mp MRI before biopsy in men with suspicion of low and intermediate risk prostate cancer.
 
Description 3D free-breathing MRI...
Amount £380,000 (GBP)
Funding ID MR/L009676/1 
Organisation Medical Research Council (MRC) 
Sector Public
Country United Kingdom
Start  
 
Description A biophysical simulation framework for magnetic resonance microstructure imaging
Amount £665,423 (GBP)
Funding ID EP/N018702/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 04/2016 
End 03/2019
 
Description A computer-guided imaging system...
Amount £10,000,000 (GBP)
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start  
 
Description Accelerated 3D Cardiac...
Amount £92,000 (GBP)
Organisation Medical Research Council (MRC) 
Sector Public
Country United Kingdom
Start  
 
Description Accelerated 3D Cardiac...
Amount £92,000 (GBP)
Organisation Medical Research Council (MRC) 
Sector Public
Country United Kingdom
Start  
 
Description Anatomy driven...
Amount £342,000 (GBP)
Funding ID EP/L022680/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start  
 
Description C-PLACID project
Amount £1,500,000 (GBP)
Funding ID EP/M006093/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 01/2015 
End 01/2018
 
Description CDT
Amount £5,690,000 (GBP)
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 01/2014 
End 09/2022
 
Description CDT (2)
Amount £4,820,000 (GBP)
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start  
 
Description Capital infrastructure for hyperpolarised MRI...
Amount £5,300,000 (GBP)
Organisation Medical Research Council (MRC) 
Sector Public
Country United Kingdom
Start  
 
Description Collaborative Award in Science application Multi-User Ultra-High Field Clinical Imaging Research Centre for London
Amount £4,000,000 (GBP)
Funding ID 201526/Z/16/Z 
Organisation Wellcome Trust 
Sector Charity/Non Profit
Country United Kingdom
Start  
 
Description Delineating impact of COVID-19 infection in high-risk populations
Amount $155,800 (USD)
Organisation Microsoft Research 
Sector Private
Country Global
Start 08/2020 
End 08/2021
 
Description Developing single-cell resolution 3D models of immune surveillance in cancer
Amount £487,000 (GBP)
Funding ID NS/A000069/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 01/2018 
End 12/2020
 
Description Dynamic High Res.
Amount £658,000 (GBP)
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start  
 
Description EPSRC Centre for Doctoral Training in Intelligent, Integrated Imaging In Healthcare (i4health)
Amount £6,034,274 (GBP)
Funding ID EP/S021930/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 09/2019 
End 03/2028
 
Description EPSRC Doctoral Prize (UCL)
Amount £110,000 (GBP)
Funding ID Andrada Ianus - two year post-doc fellowship 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 01/2016 
End 12/2017
 
Description EPSRC Early career fellowship
Amount £1,000,000 (GBP)
Funding ID EP/N021967/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 07/2016 
End 06/2021
 
Description ERC Synergy Grant
Amount € 15,000,000 (EUR)
Organisation European Research Council (ERC) 
Sector Public
Country Belgium
Start  
 
Description Enabling Clinical Decisions From Low-power MRI In Developing Nations Through Image Quality Transfer
Amount £1,035,545 (GBP)
Funding ID EP/R014019/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 02/2018 
End 01/2021
 
Description Enabling clinical decisions from low-power MRI in developing nations through image quality transfer
Amount £1,020,000 (GBP)
Funding ID EP/R014019/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 02/2018 
End 01/2021
 
Description Exploiting... PET/MRI
Amount £1,270,000 (GBP)
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 06/2013 
End 10/2016
 
Description FP7 (PICTURE)
Amount € 735,000 (EUR)
Organisation European Commission 
Sector Public
Country European Union (EU)
Start 02/2013 
End 01/2016
 
Description FP7 (PRISM)
Amount € 738,000 (EUR)
Funding ID 316746 
Organisation European Commission 
Sector Public
Country European Union (EU)
Start 03/2013 
End 02/2016
 
Description Gold-standard assessment of prostate cancer MRI accuracy
Amount $25,000 (AUD)
Organisation Sydney Catalyst 
Sector Charity/Non Profit
Country Australia
Start 06/2017 
End 05/2018
 
Description HICF
Amount £1,850,000 (GBP)
Organisation Health Innovation Challenge Fund 
Sector Charity/Non Profit
Country United Kingdom
Start  
 
Description HICF-Smart laparoscopic
Amount £1,850,000 (GBP)
Organisation Health Innovation Challenge Fund 
Sector Charity/Non Profit
Country United Kingdom
Start 11/2012 
End 10/2016
 
Description HICF-SmartTarget
Amount £1,850,000 (GBP)
Organisation Health Innovation Challenge Fund 
Sector Charity/Non Profit
Country United Kingdom
Start 10/2012 
End 10/2016
 
Description HTC Network
Amount £1 (GBP)
Organisation National Institute for Health Research 
Sector Public
Country United Kingdom
Start  
 
Description ICR Cancer Imaging Centre
Amount £7,600,000 (GBP)
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start  
 
Description INNOVATE
Amount £307,000 (GBP)
Organisation Prostate Cancer UK 
Sector Charity/Non Profit
Country United Kingdom
Start  
 
Description Image-guided Intrauterine...
Amount £9,990,000 (GBP)
Organisation Wellcome Trust 
Sector Charity/Non Profit
Country United Kingdom
Start 07/2014 
End 06/2021
 
Description In vivo microstructural...
Amount £794,000 (GBP)
Organisation Medical Research Council (MRC) 
Sector Public
Country United Kingdom
Start  
 
Description Interdisciplinary Prize
Amount £249,000 (GBP)
Organisation Rosetrees Trust 
Sector Charity/Non Profit
Country United Kingdom
Start 01/2017 
End 12/2019
 
Description JPND: Stratification of presymptomatic amyotrophic lateral sclerosis: the development of novel imaging biomarkers
Amount € 1,600,000 (EUR)
Funding ID MR/T046473/1 
Organisation JPND Research 
Sector Academic/University
Country Global
Start 07/2020 
End 07/2023
 
Description KCL/UCL Cancer Imaging Centre
Amount £8,000,000 (GBP)
Organisation Cancer Research UK 
Sector Charity/Non Profit
Country United Kingdom
Start 12/2013 
End 11/2018
 
Description Learning MRI and histology image mappings for cancer diagnosis and prognosis
Amount £774,000 (GBP)
Funding ID EP/R006032/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 12/2017 
End 01/2020
 
Description MIMIC project
Amount £812,000 (GBP)
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 04/2013 
End 07/2013
 
Description Marie Curie Fellowship
Amount € 197,000 (EUR)
Organisation European Commission 
Sector Public
Country European Union (EU)
Start  
 
Description Marie Curie Fellowship - towards the simulation of Breast...
Amount € 231,000 (EUR)
Funding ID 627025 
Organisation European Commission 
Sector Public
Country European Union (EU)
Start 02/2015 
End 01/2017
 
Description Medical Bioinformatics Computer Infrastructure
Amount £6,800,000 (GBP)
Organisation Medical Research Council (MRC) 
Sector Public
Country United Kingdom
Start  
 
Description Medical Image Computing ... (Platform)
Amount £1,400,000 (GBP)
Funding ID EP/M020533/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 04/2015 
End 03/2020
 
Description Multidisciplinary Project Award
Amount £499,692 (GBP)
Funding ID C33589/A19908 
Organisation Cancer Research UK 
Sector Charity/Non Profit
Country United Kingdom
Start 07/2015 
End 06/2018
 
Description NIHR-I4I (MRI-augmented...)
Amount £410,000 (GBP)
Organisation National Institute for Health Research 
Department NIHR i4i Invention for Innovation (i4i) Programme
Sector Public
Country United Kingdom
Start  
 
Description National facility for in vivo MRI...
Amount £2,900,000 (GBP)
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start  
 
Description Next generation MRI
Amount £1,500,000 (GBP)
Organisation Medical Research Council (MRC) 
Sector Public
Country United Kingdom
Start  
 
Description PET-MR Motion correction...
Amount £590,000 (GBP)
Funding ID EP/M009319/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start  
 
Description Placenta Imaging Project
Amount $3,229,581 (USD)
Funding ID 1U01HD087202-01 
Organisation National Institutes of Health (NIH) 
Sector Public
Country United States
Start 09/2015 
End 09/2018
 
Description Real-time MRI guided radiation therapy...
Amount £200,000 (GBP)
Organisation Cancer Research UK 
Sector Charity/Non Profit
Country United Kingdom
Start  
 
Description S.Malik Fellowship
Amount £565,000 (GBP)
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start  
 
Description Simultaneous PET-MR...
Amount £696,000 (GBP)
Funding ID EP/M020142/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start  
 
Description Synergistic PET-MR...
Amount £261,000 (GBP)
Funding ID EP/M022587/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start  
 
Description TSB - Guide CRT
Amount £1,000,000 (GBP)
Funding ID 101681 
Organisation TSB Bank plc 
Sector Private
Country United Kingdom
Start  
 
Description UCLh/UCLH Computational Imaging Infrastructure
Amount £915,000 (GBP)
Organisation National Institute for Health Research 
Department NIHR Biomedical Research Centre
Sector Public
Country United Kingdom
Start 10/2013 
End 03/2017
 
Description Using Machine Learning...
Amount £601,000 (GBP)
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start  
 
Description Wellcome Centre
Amount £12,100,395 (GBP)
Funding ID 203148/Z/16/Z 
Organisation Wellcome Trust 
Sector Charity/Non Profit
Country United Kingdom
Start 06/2017 
End 07/2022
 
Title 3D-printed patient-specific prostate molds 
Description We developed a mold template that contains landmark features to guide MRI scanning and guides for histological slicing. Using contours from in vivo MRI scans, the mold can be designed to hold the prostate in the in vivo position and orientation following surgical removal. This allows in vivo images to be compared with those before and after fixation, as well as histological images in the same slice plane. 
Type Of Material Biological samples 
Provided To Others? No  
Impact We are using this tool to validate biomarkers from diffusion MRI using corresponding histology. The device can also be applied for validation of other MRI techniques. We are consulting with the surgical and pathological teams on the use of the device to guide histological slicing and confirm patient margins. An additional EPSRC grant has been submitted on the use of machine learning techniques to relate radiological features with histological cancer grade that would benefit from this technique. 
 
Title Camino software 
Description This is an open source package for diffusion MRI processing. 
Type Of Material Improvements to research infrastructure 
Year Produced 2012 
Provided To Others? Yes  
Impact This software is very widely used in the diffusion MRI research community. 
URL https://sourceforge.net/projects/camino/files/latest/download
 
Title MRI methods research software 
Description Source code for MRI methods developed as part of research project 
Type Of Material Improvements to research infrastructure 
Year Produced 2016 
Provided To Others? Yes  
Impact Dissemination of research software that is being used by others in the field 
URL http://mriphysics.github.io/
 
Title Nifty open source software 
Description A series of open source software packages for image registration (niftyreg), numerical simulation/modelling (niftysim), image segmentation (niftyseg). Details at: http://cmictig.cs.ucl.ac.uk/wiki/index.php/Main_Page 
Type Of Material Improvements to research infrastructure 
Year Produced 2012 
Provided To Others? Yes  
Impact The tools, in particular Niftyreg have been very widely downloaded by the imaging community (academic, medical users and industry). Niftysim arose directly from the Intelligent Imaging Programme Grant. 
URL http://cmictig.cs.ucl.ac.uk/wiki/index.php/Main_Page
 
Title Summary of Progress (as in final report submitted to EPSRC) 
Description Summary of Progress: We have made significant progress in developing new methods for compensating for respiratory motion, cardiac motion and fetal motion to improve image quality and hence diagnosis, fetal interventions and guide radiotherapy in the lung. Our new methods exploit sparsity and redundancy to speed up image acquisition and reconstruction. We have devised new ways to compensate for motion and provide fast reconstruction of MR images of the adult heart and fetus based on manifold learning and super-resolution using dictionary learning. Manifold learning has also been used learn respiratory motion. We have developed a generalised framework that combines motion model fitting with motion compensated image reconstruction. Our methods have been applied to MR imaging and ultrasound imaging in the heart and fetus and CT imaging in the lungs as well as respiratory compensation in PET imaging. An exciting new innovation is the application of related methods to a novel photoacoustic imaging device, potentially for assessing tissue microstructure and vascularity for minimally invasive surgery. While measuring and compensating for motion is a core theme of the programme it is important that the effects of motion are minimised as much as possible at acquisition. We have adapted an MR compatible Active Breathing Coordinator (ABC) originally developed for radiotherapy to minimise patient breathing motion during MR acquisition. Our pioneering work on microstructure imaging, VERDICT, coupled with image guided biopsy and focal ablation is looking to drive radical changes in the detection, stratification and management of prostate cancer. Related tissue microstructure work is shedding light on how the fetus develops and how we might improve detection of fetal abnormalities and interventions to mitigate them. The work on fetal and neonatal imaging is providing new insight into how the normal brain develops and how this normal development might be disrupted, for example by oxygen starvation. An image guided laparoscopic surgery system has been developed with funds from HICF (Wellcome and DoH) Smart Surgery programme (see follow on funding). Several innovative components have been incorporated into this system as a result of work initiated by the Programme Grant (see papers with first author Johnsen, Clarkson, Song, Thompson, Totz in the publication list). The system has been translated into clinical with a trial on 10 patients (so far) undertaken. The system provides the core technology for planned developments in laparoscopic surgery (e.g. pancreas, kidney, bowel etc etc) and forms a key component of the recently awarded Wellcome Trust Centre for Surgical and Inreventional Sciences. Commercialisation: Graeme Penney has spun out Cydar to commercialise aspects of visualisation in vascular surgery by providing 3D overlays to improve clinicians' perception of 3D anatomy. (www.cydar.co.uk) Seb Ourselin has spun out Brain Miner, a data mining system for assessing disease progression in dementia. Dean Barratt has spun out Smart Target, a system for providing image guidance to prostate biopsy and focal ablation Maria Panayiotou's work with Kawal Rhode on motion compensation is being integrated into a commercial prototype with Siemens as part of an InnovateUK grant for £1M called GuideCRT. Philips Research sponsored a PhD student with Dave Hawkes on modelling breast deformation for alignment of images of the breast and image guided breast surgery. Philips Research are funding a PhD studentship through the KCL CDT related to learning motion-based biomarkers of cardiac function from MRI and an EPSRC grant held by Andy King that follows on from the Programme Grant. Elekta will sponsor an MR-ABC project with ICR. Elekta are sponsoring an EngD student with Jamie McClelland and Dave Hawkes at UCL on integration of motion compensated cone-beam CT reconstruction and tumour tracking in lung radiotherapy and are supporting work between ICR and UCL on development of motion compensation in the combined MR-Linac system. Vision-RT sponsored an EngD student with Jamie McClelland and Dave Hawkes at UCL on the development of motion management strategies in radiotherapy delivery. On the dynamic photoacoustic imaging project UCL filed patent PCT/GB2014/051910 "Apparatus and Method for Performing Photoacoustic Tomography", June 2013. Clinical Translation While the basis of our Programme is development of novel methodology all the work that we do has a clinical application as its driver. As stated above we are in a strong position to translate our methodological work to clinical trial, and hence to commercialisation and clinical practice, through our strong links to our respective NIHR Biomedical Research Centres who are now investing significantly in that translation. Examples relevant to the Programme Grant include: An initial clinical trial on the VERDICT was completed last year and the results have been published (Panagiotaki et al 2015). A much larger trial, "INNOVATE", on 450 patients has been funded by the medical charity Prostate Cancer UK and is about to start. As part of PROMIS, and other large scale clinical trials of management of early stage prostate cancer, the image guided biopsy and focal ablation technologies are in large scale clinical trial (several 100s of patients). The PROMIS trial, in which we had input, has now reported (Ahmed et al Lancet 2017) and is already having a major impact on national and intenational guidelines on prostate cancer detection and management. The core finding is that over 25% of men with low and intermediate disease can safely avoid biopsy by having a multi-parametric MR to rule-out significant disease prior to biopsy. A further 18% will have significant disease detected by pre-biopsy mp MRI that woudl have been missed by conventional transrectal biopsy. An initial first-in-man clinical trial of the MR-ABC for respiratory motion control is underway at ICR Muhammad Usman's work has translated to clinical trial in exercise CMR in adult congenial heart patients at KCL. Radomir Chabiniok's work is being used in KCL's heart failure patient cohort. Joshua van Amerom's work on fetal CMR is translating into clinical trial at KCL. The prototype image guided liver surgery system has entered initial clinical trial at the Royal Free Hospital with data collected on 10 patients to date. 
Type Of Material Data analysis technique 
Year Produced 2012 
Provided To Others? Yes  
Impact The work of the programme, in particular imaging moving structures has fed in to two major programmes at KCL and Imerial and at UCL in Fetal Imaging and Fetal Interventions resulting in several large awards (see follow-on funding). It has also fed into commercial development in cardiac imaging and interventions (Philips and Siemens). The respiratory motion work has fed into major developments with Elekta and Philips as part of the combined MR-Linac system with a CRUK/EPSRC Multidisciplinary award and commercial sponsorship. It has also formed a major component of the Cancer Research UK ART-NET Network Accelerator award. The prostate work has contributed to the PROMIS trial and other prostate imaging and interventional trials. The PROMIS trial has recently reported (Ahmed et al Lancet 2017) and already has had a major impact on guidelines for teh detection and management of low and intermediate risk prostate cancer. This common disease (~160000 new cases per annum in the UK according to Cancer Research UK statistics) effects a large number of men over 50. The new findings have shown that with a negative multiparametric MRI more than 25% of men with low and intermediate risk disease can avoid having a painful and potentially risky transrectal biopsy, while a further 18% will be detected with clinically significant disease that otherwise woudl have been missed. The VERDICT trial arising from our programme is looking to improve the sensitivity and specificity of MRI even further. The significant number of publications resulting from this programme are listed above. 
 
Title Synthetic data for Baumgartner et al, MedIA 2017 
Description Real 2D T1 weighted dynamic MR data acquired during free breathing 
Type Of Material Database/Collection of data 
Year Produced 2016 
Provided To Others? Yes  
Impact Unknown 
URL https://zenodo.org/record/55345#.WJh7Z1OLSUl
 
Description Atlantic Consortium 
Organisation Elekta Inc
Country Sweden 
Sector Private 
PI Contribution We are a site for testing the first clinical MR-Linacs, involving a substantial investment here in infrastructure and equipment. We bring excellence in imaging, image guided radiotherapy and oncology.
Collaborator Contribution UMCU, MDACC, NKI, Sunnybrook, F&MCW, ICR-RMH, MCRC & The Christie are clinical/university partners in this initiative. Electa and Philips are the commercial partners who are developing the equipment. We will distribute applications to different clinical problems across the partners, and address the many aspects of developing and implementing this new approach.
Impact CRUK grants, MRC grant, MR Linac installed at our hospital Multi disciplinary physics, oncology, radiotherapy, engineering, clinical trials, radiography, imaging, radiology
Start Year 2014
 
Description Atlantic Consortium 
Organisation Philips Healthcare
Country Netherlands 
Sector Private 
PI Contribution We are a site for testing the first clinical MR-Linacs, involving a substantial investment here in infrastructure and equipment. We bring excellence in imaging, image guided radiotherapy and oncology.
Collaborator Contribution UMCU, MDACC, NKI, Sunnybrook, F&MCW, ICR-RMH, MCRC & The Christie are clinical/university partners in this initiative. Electa and Philips are the commercial partners who are developing the equipment. We will distribute applications to different clinical problems across the partners, and address the many aspects of developing and implementing this new approach.
Impact CRUK grants, MRC grant, MR Linac installed at our hospital Multi disciplinary physics, oncology, radiotherapy, engineering, clinical trials, radiography, imaging, radiology
Start Year 2014
 
Description Berlin 
Organisation Charité - University of Medicine Berlin
Department Institute of Radiology
Country Germany 
Sector Hospitals 
PI Contribution Image registration technologies for alignment of X-ray mammograms with breast MRI images
Collaborator Contribution Carefull assessment of the results of our alignment software through a radiological observer study
Impact Alignment software and a clinical validation for establishing correspondence between X-ray mammograms and breast MR images (Martzanidou et al 2012 and 2014).
Start Year 2008
 
Description Dundee Radiology 
Organisation University of Dundee
Country United Kingdom 
Sector Academic/University 
PI Contribution As part of this EPSRC project we provided advice and assistance on diffusion MRI sequences for breast MR imaging; advice as how to prepare lumpectomy samples for comparision between post-op histology and pre-op MR imaging; input to analysis of ultrasound shear wave data including results from our numerical modelling work.
Collaborator Contribution Our Dundee partners, Professor Andy Evans and Dr Sarah Vinnicombe, provided multi-parametric MR images including multiple b-value diffusion imaging to our specification on 19 patients. They also provided access to shear wave ultrasound images on 1137 tumours from 1112 women. Both sides contributed in a major way to the development of the ideas and the study protocols of the project.
Impact A number of research presentations and journal papers have arisen from this work. Please see the reference list. The Impact has been to furthering the clinical and scientific goals of our project. There doesn't seem to be a box available for that below.
Start Year 2012
 
Description Elekta 
Organisation Elekta Inc
Department Elekta UK
Country United Kingdom 
Sector Private 
PI Contribution Elekta have sponsored 2 PhD students as part of a Master Research Agreement with UCL. These students have been developing motion compensated cone-beam CT reconstructions to better provide image guided radiotherapy. See publication list.
Collaborator Contribution Funding of the project and access to Elekta's expertise in radiotherapy equipment, the linear accelerator and its control.
Impact See publications above.
Start Year 2013
 
Description Frauhnhofer Mevis 
Organisation Fraunhofer Society
Department Fraunhofer MEVIS
Country Germany 
Sector Charity/Non Profit 
PI Contribution As partners in two EU projects (HAMAM and PRISM) we have supplied expertise in image registration, biomechanical modelling and MR diffusion imaging
Collaborator Contribution Mevis provided expertise in software developments, clinical translation, visualisation and image alignment.
Impact Various papers and conference publications (see publication list). Mevis produced several radiology systems for breast image integration, analysis and visualisation in which we had input.
Start Year 2008
 
Description Guys Pathology Collaboration 
Organisation King's College London
Department Division of Imaging Sciences and Biomedical Engineering
Country United Kingdom 
Sector Academic/University 
PI Contribution We have worked closely with Professor Sarah Pinder and her team in breast pathology. We have provided the output of our work which is an academic image analysis system for assessing peritumoral breast stroma maturity. We have worked closely to together to devise a novel image analysis system for undertaking this task (Reis et al 2017). We have assessed the VERDICT MR diffusion analysis on the 7 breast cancer samples supplied by KCL. Data has been correlated accurately with histology. Detailed analysis of the diffusion signal and it's anistropy has enabled significant advances in our understanding and interpretation of the MR diffusion signal in terms of tissue microstructure (Bailey et al 2017).
Collaborator Contribution Sarah Pinder has provided invaluable advice and expertise in pathological image interpretation. She and her team have undertaken a significant observer study to evaluate our methods. They have also supplied us with 7 breast cancer samples from the King's Health Partners Cancer Biobank.
Impact This is a multidisicplinary collaboration between breast pathology and medical image computing. Several papers have resulted from this work - see publication list. Again the impact is scientific and medical - not one of the boxes below.
Start Year 2012
 
Description IRCAD IHU 
Organisation IRCAD France
Country France 
Sector Hospitals 
PI Contribution I am a member of the IRCAD IHU Scientific Advisory Board and provide advice on their programme as well as input from our own research activity.
Collaborator Contribution Being exposed to the work of the IRCAD IHU has opened a number of opportunities.
Impact Several of our projects in computer assisted surgery have benefitted from this link, in particular the Smart Surgery image guided laparoscopic liver surgery project, in which IRCAD is a formal partner.
Start Year 2012
 
Description Lisbon 
Organisation University of Lisbon
Country Portugal 
Sector Academic/University 
PI Contribution We have provided visualisation, image analysis and image registration expertise to the breast surgery group in Lisbon as part of teh PICTURE project.
Collaborator Contribution Our partner in Lisbon has supplied imaging data and other information, as well as interpretation of the results of our wound healing simulation.
Impact The main output has been production of accurate biomechanical models that establish correspondence between prone, supine and upright views of the female breast (Bjoern et al 2016, Han et al 2014) and development a prototype system for predicting breast deformation due to wound healing post lumpectomy (Vavourakis et al 2016).
Start Year 2013
 
Description Multi-scale Computational Anatomy 
Organisation Nagoya University
Country Japan 
Sector Academic/University 
PI Contribution I have provided expertise as a member of the international scientific advisory board to the Japanese Multi-scale Computational Anatomy Programme
Collaborator Contribution I have had access to their technology and research programme which has benefitted several of our programmes, but in particular the Intelligent Imaging EPSRC Programme. In 2015 I was funded by Nagoya on a 3 month sabbatical.
Impact A direct outcome has been a series of papers on image guided gastric surgery (see publications).
Start Year 2015
 
Description Nijmegen 
Organisation Radboud University Nijmegen Medical Center
Department Department of Radiology and Nuclear Medicine
Country Netherlands 
Sector Hospitals 
PI Contribution We established a collaboration with Nijmegen which led to two EU projects HAMAM and PRISM. PRISM arose during the duration of the Programme Grant and the EPSRC MIMIC grant. We contributed expertise and work on image registration and biomechanical modelling.
Collaborator Contribution Nijmegen provide a significant amount of data for both EU projects. They also provided expertise in pathological specimen preparation post mastectomy. Together we developed a method for alignment of histology specimens and medical image data (Mertzanidou et al 2017). Nijmeneg also provided significant expertise in breast imaging and breast cancer screening and diagnosis.
Impact A method for preparation of breast tissue samples for histology and alignment of the resulting data with medicall imaging (Mertzanidou et al 2017). Methods for establishing correspondence between x-ray mammogram breast screening images and MRI (Mertzanidou et al 2014, Mertzanidou et al 2012). A method for characterising stromal maturity and breast cancer risk from H&E stained histological images (Reis et al 2017). This is multi-disciplinary involving computer scientists, physicists, breast pathologists and radiologists.
Start Year 2008
 
Description Philips Research labs Hamburg 
Organisation Koninklijke Philips Electronics N.V.
Department Philips Research Hamburg
Country Germany 
Sector Private 
PI Contribution We have worked with Philips for well over 20 years. For the period covered by the Programme Grant and EPSRC MIMIC we have had two PhD students sponsored by Philips and in-kind support from the team in Hamburg. We have contributed our work and expertise in image registration, image analysis and in particular in biomechanical modelling of breast tissue deformation during the significant manipulations of the breast (e.g. compression) arising during imaging.
Collaborator Contribution Input on the business objectives of Philips. Significant help and expertise in mechanical modelling and image acquisition.
Impact Over the years the output from this collaboration have been substantial and game-changing, in particularly in the area of image registration with some of the most highly cited papers in our field resulting from the joint working (e.g papers by Rueckert, Penney and Studholme between 1996 and 2000). In the time of the Programme grant, MIMIC and the three EU projects HAMAM, MIMIC and in particular PICTURE we have developed innovative technology and published key papers on alignment of X-ray mammograms and MR image (Mertzanidou et al 2012, 2014), produced accurate biomechanical models that establish correspondence between prone, supine and upright views of the female breast (Bjoern et al 2016, Han et al 2014), developed a prototype system for predicting breast deformation due to wound healing post lumpectomy (Vavourakis et al 2016), developed a microstructural model of collagen re-alignment to predict peritumoral stromal changes (Wijeratne et al 2016), developed a system to assess peritumoral stromal maturity in H&E stained histology images of the breast (Reis et al 2017), shown that diffusion MRI can detect stromal anisotropy in breast cancer tissue samples (Bailey et al 2017).
 
Description Roger Bourne 
Organisation University of Sydney
Department School of Pharmacy and Medical Sciences
Country Australia 
Sector Academic/University 
PI Contribution Application of the system developed in Sydney to the prostatectomy specimens collected at UCLH
Collaborator Contribution Access to expertise and experience in developing the system for tissue sample holding and slice preparation of whole prostatectomy specimens.
Impact A methods paper has just been published and the method has been adopted by the prostate group to allow accurate co-registration of clinical and histological imaging.
Start Year 2016
 
Description Vision RT 
Organisation Vision RT Ltd.
Country United Kingdom 
Sector Private 
PI Contribution We have over the years provided Vision RT with access to our expertise in image registration and image guided radiotherapy. From the start of the company in ~2000 I have been a member of their scientific advisory board. We undertook the initial work assessing the accuracy of their core technology, a vision based positioning system.
Collaborator Contribution Vision RT have provided very useful insight over teh years to the unmet needs in radiotherapy and access to a range of technical solutions that their company has offered. They sponsored 2 PhD students over this time and loaned us a number of versions of their technology for our research.
Impact See publications list - in particular papers with James Martin and Jamie McClelland as first author.
 
Title APPARATUS AND METHOD FOR PERFORMING PHOTOACOUSTIC TOMOGRAPHY 
Description A method and apparatus are provided for performing photoacoustic tomography with respect to a sample that receives a pulse of excitation electromagnetic radiation and generates an acoustic field in response to said pulse. One embodiment provides an apparatus comprising an acoustically sensitive surface, wherein the acoustic field generated in response to said pulse is incident upon said acoustically sensitive surface to form a signal. The apparatus further comprises a source for directing an interrogation beam of electromagnetic radiation onto said acoustically sensitive surface so as to be modulated by the signal; means for applying a sensitivity pattern to the interrogation beam; and a read-out system for receiving the interrogation beam from the acoustically sensitive surface and for determining a value representing a spatial integral of the signal across the acoustically sensitive surface, wherein said spatial integral is weighted by the applied sensitivity pattern. The apparatus is configured to apply a sequence of sensitivity patterns to the interrogation beam and to determine a respective sequence of values for said weighted spatial integral for generating a photoacoustic image. 
IP Reference WO2014207440 
Protection Patent application published
Year Protection Granted 2014
Licensed No
Impact On going research to further develop the system. Significant impact is expected in providing real-time 3D photo acoustic imaging for image guided interventions
 
Title APPARATUS AND METHOD FOR REGISTERING PRE-OPERATIVE IMAGE DATA WITH INTRA-OPERATIVE LAPARSCOPIC ULTRASOUND IMAGES 
Description A method and apparatus are provided for registering pre-operative three dimensional (3- D) image data of a deformable organ comprising vessels with multiple intra-operative two- dimensional (2-D) ultrasound images of the deformable organ acquired by a laparoscopic ultrasound probe during a laparoscopic procedure. The apparatus is configured to: qenerate a 3-D vessel graph from the 3-D pre-operative image data; use the multiple 2-D ultrasound images to identify 3-D vessel locations in the deformable organ; determine a rigid registration between the 3-D vessel graph from the 3-D pre-operative image data and the identified 3-D vessel locations in the deformable organ; and apply said rigid registration to align the pre- operative three dimensional (3-D) image data with the two-dimensional (2-D) ultrasound images, wherein the rigid registration is locally valid in the region of the deformable organ of interest for the laparoscopic procedure. 
IP Reference WO2016170372 
Protection Patent application published
Year Protection Granted 2016
Licensed No
Impact First example of a non-contact image guidance system for laparoscopic surgery.
 
Title VERDICT 
Description VERDICT is currently undergoing clinical trial at UCLH and is still recruiting 
Type Diagnostic Tool - Imaging
Current Stage Of Development Early clinical assessment
Year Development Stage Completed 2015
Development Status Under active development/distribution
Impact We are exploring whether VERDICT could further reduce the need for biopsy in low risk prostate cancer and improve assessment of prostate cancer risk prior to biopsy. 
 
Company Name Smart Target 
Description Provides a system that uses MR images of the prostate to guide transperineal biopsy and image guided focal ablation. 
Year Established 2016 
Impact Selected for a significant trial of a novel needle based therapy for prostate cancer. Provides the most accurate overlay available of MR derived targets on transrectal ulatrsound. This is particular important given the findings of the recent PROMIS trial and the proposed role of MRI in the detection of significant cancer.
Website http://www.smarttarget.co.uk/
 
Company Name Cydar 
Description Supplies software to provide guidance in minimally invasive aortic stent placement 
Year Established 2016 
Impact The first system that provides guidance in realtime from 3d models derived from CT angiography during fluoroscopy assisted aortic stent placement
Website http://cydarmedical.com
 
Description MR-Linac 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact A press release is being prepared on the Cancer Research UK/EPSRC funded mutlidisicplinary project on developing image guidance strategies and in particular compensation for respiratory motion in teh new combined MR-Linac image guided radiotherapy system being developed by Philips and Elekta.
Year(s) Of Engagement Activity 2017
 
Description Media engagement 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Interviews, Blogging, Tweets
Year(s) Of Engagement Activity 2015,2016,2017
URL http://www.ifindproject.org
 
Description Prostate Cancer UK 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Patients, carers and/or patient groups
Results and Impact For the last 12 months Dave Hawkes has been involved in fund raising and public awareness of the UCL/UCLH research for Prostate Cancer UK. An article and press release on the patient story and participation in reasearch is in preparartion.
Year(s) Of Engagement Activity 2017