A framework for efficient synergistic spatiotemporal reconstruction of PET-MR dynamic data

Lead Research Organisation: University College London
Department Name: Medicine

Abstract

Magnetic resonance (MR) and radionuclide imaging using positron emission tomography (PET) are now essential in many areas of medical diagnosis, clinical research and drug development. The complementary information provided by PET and MR has recently lead to a new generation of dual-modality systems. Clinical applications of these devices are emerging in many priority areas for human health, including dementia, cardiology, and oncology.
The previously established Collaborative Computational Project (CCP) for synergistic PET-MR reconstruction connects researchers working on sophisticated algorithms to estimate stacks of slices through the patient from the measured data. In this flagship project, we extend this work towards imaging of dynamic processes. Measuring the redistribution over time of contrasts agents or labelled molecules while taking organ movement into account provides considerable information on the biology, allowing extraction of parameters such as blood flow, receptor density and tissue elasticity.
By exploiting correlations between these parameters in both space and time, we can probe fast dynamic processes at higher accuracy than achievable in single modality imaging. This flagship will provide freely available software and methods to the community to accelerate research and development in this area.

Planned Impact

Biomedical imaging plays a key role in patient management, the assessment of new therapies and improved understanding of disease. PET-MR imaging is a newly developed technology that may provide new information that increases the effectiveness of medical imaging.

Patients
In the long-term, new imaging can benefit patients through more accurate and earlier diagnosis, and assessment of response to therapy. An early assessment of therapy response allows for a change in dose or management to match the response of the patient. This could lead to effective treatment occurring faster, or prevent prolonged unnecessary treatments with negative side-effects.

Health Service
Improved diagnosis and monitoring of disease and therapy response allows a more rational use of healthcare resources. Although imaging costs for PET-MR are of the order £1K/scan, the potential saving in costs of drugs or other interventions through better and earlier decisions is likely to far exceed this cost. In the UK, the NHS spends only about 7% of its budget on imaging.

Pharmaceutical Industry & Advanced imaging centres
The combined use of PET and MR offers the possibility of making measurements from PET and MR images more reliable and reproducible between different patients and imaging centres. Changes within a patient's images can be used as biomarkers for disease, potentially showing disease changes before symptoms are manifest. These quantitative imaging biomarkers are used in drug development and thus our methods will be useful to the Pharmaceutical industry and Advanced Imaging Centres. Improved quantification and accuracy will also improve statistical power in clinical trials, potentially allowing studies using smaller numbers of subjects or able to reach significant conclusions earlier.

The clinical imaging industry
Impact is envisaged via the adoption of techniques we develop into the continuous improvement of commercial scanners by the main manufacturers (General Electric, Siemens and Philips). Industry representatives attend the Collaborative Computational Project (CCP) which is integrated with this project and our results and algorithms are freely available with a software licence that permits commercial use. Whilst aimed at dual-modality PET-MR systems, some of our techniques will be applicable to single modality systems which currently have a much larger market share.

The preclinical imaging industry
Major leading companies such as General Electric and Siemens who were originally leading the preclinical PET-MR imaging industry have disengaged due to lack of financial growth in the preclinical imaging market. Three main preclinical PET-MR scanner developers are currently leading the market: Mediso, Bruker and MRsolutions. These are relatively medium size enterprises which strive to resource image reconstruction due to the complexity and resource demands involved. It is harder for these small preclinical PET-MR companies to maintain and extend advanced software, which needs to include detailed modelling of physical and physiological phenomena. Thus, open access of reconstruction software will foster the preclinical PET-MR imaging sector.

Publications

10 25 50
 
Description This Software Flagship project contributes heavily to the open source software SIRF (Synergistic Image Reconstruction Framework) and its underlying component STIR (Software for Tomographic Image Reconstruction) which is used for PET reconstruction. It also contributes various fixes and enhancements to other supporting packages. The contributions of this project concentrate on image registration, motion correction, simulation, but also increasing support for various file formats to be able to compare images with DICOM images from the scanner.

We are currently evaluating a method for PET/MR head motion correction that uses both MR and PET data for motion detection and estimation in separate parts of the acquisition. Part of this work has been presented at international conferences.

In collaboration with CSIRO we have developed a novel method for continuous head pose estimation from PET data (using a surrogate signal obtained from the data and various position estimates). This work was presented during an oral presentation at the IEEE Medical Imaging Conference 2019 to excellent response.

Taken together, this work has the potential to remove the need for external monitoring equipment for head motion, simplifying set-up and workflow, especially important in PET/MR.
Exploitation Route The methods developed here will become part of our open source package SIRF for others to explore and validate.
Sectors Digital/Communication/Information Technologies (including Software),Healthcare

 
Description CCP on Synergistic Reconstruction for Biomedical Imaging
Amount £464,610 (GBP)
Funding ID EP/T026693/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 04/2020 
End 03/2025
 
Description PhD Studentship on PET Motion Correction
Amount £100,000 (GBP)
Organisation GE Healthcare Limited 
Sector Academic/University
Country United Kingdom
Start 10/2018 
End 03/2022
 
Description Whole Body dynamic PET
Amount £252,000 (GBP)
Organisation GE Healthcare Limited 
Sector Academic/University
Country United Kingdom
Start 02/2020 
End 02/2022
 
Description Data-driven methods for head motion correction in PET/MR 
Organisation Commonwealth Scientific and Industrial Research Organisation
Country Australia 
Sector Public 
PI Contribution Open source software for PET/MR image reconstruction (SIRF and STIR).Training in use and development of the software. Expertise in PET data-driven motion detection techniques. Financial assistance with travel and subsistence of PhD student.
Collaborator Contribution Contributions to the STIR open source software for handling data of the Siemens PET/MR scanner, with specific attention on co-registration between PET reconstructed by STIR and the scanner software. Methods for continuous head position estimation based on the PET data only.
Impact Several pull request to STIR open source software. Oral presentation at IEEE Medical Imaging Conference 2019.
Start Year 2018
 
Description Motion correction for cardiac PET/MR 
Organisation Physikalisch-Technische Bundesanstalt
Country Germany 
Sector Academic/University 
PI Contribution Our main contribution is via the open source software SIRF and associated software for PET reconstruction STIR, enabling reconstruction of data from the Siemens mMR PET/MR scanner in an independent and open framework, allowing offline processing, non-standard gating etc. We have provided financial support for visits by a PhD student and training in the software and development tools. In addition, we bring expertise on respiratory motion correction in PET/MR.
Collaborator Contribution MR expertise and test-data. In addition, Dr Kolbitsch and his PhD student have made substantial contributions to the training material for the software, as well as actively participating in training events. They are in the process of contributing an addition to our SIRF software allowing simulation of dynamic PET/MR with motion, which will be a major step towards giving researchers easy-to-use tools for developing and validating advanced motion correction strategies.
Impact jupyter notebooks for MR reconstruction via SIRF, see https://github.com/CCPPETMR/SIRF-Exercises. Paper accepted in Phys Med Biol on the simulation framework. Extra capabilities for SIRF are in development and will be contributed in the near future.
Start Year 2018
 
Description Software collaboration with the Gadgetron project for Open Source software for MR image reconstruction 
Organisation National Heart, Lung, and Blood Institute (NHLBI)
Country United States 
Sector Public 
PI Contribution Our contributions have been small at present. We have contributed a few bug fixes and documentation enhancements. We are also suggesting alternative ways for building the interface between the Gadgetron and Python and MATLAB, based on our own work.
Collaborator Contribution The Gadgetron is one of the best-established open source projects for MR image reconstruction. It is able to reconstruct data from many clinical systems. Moreover, we have had discussions on software architecture that help us in building a computationally efficient tool for the research community.
Impact As our CCP will develop synergistic PET-MR reconstruction, being able to use the Gadgetron is a large time and money saver in terms of man-years, but also in terms of available expertise.
Start Year 2015
 
Title CCP 
Description Open Source software for reconstruction of PET and MR raw data. Developed under an EPSRC Collaborative Computing Project. 
Type Of Technology Software 
Year Produced 2017 
Open Source License? Yes  
Impact Further funding for a 'flagship' application. 
URL http://www.ccppetmr.ac.uk
 
Title SIRF v2.0 
Description SIRF (Synergistic Image Reconstruction Framework) wraps various open source projects in one consistent C++/Python/MATLAB framework. v2.0 adds image registration and resampling as a basis for motion correction. 
Type Of Technology Software 
Year Produced 2019 
Open Source License? Yes  
Impact This software is still in relatively early stage but is already enabling researchers in UK, Germany and Australia to perform research on novel methods for processing of PET/MR data. 
URL http://www.ccppetmr.ac.uk/
 
Title SIRF v2.1 
Description SIRF (Synergistic Image Reconstruction Framework) wraps various open source projects in one consistent C++/Python/MATLAB framework. v2.1 adds integration with the CCPi Core Imaging Library (CIL), an interface to the Hybrid Kernel Method for PET image reconstruction using anatomical information from MR, and capability of reconstruction of 3D MR sequences. This new version makes the software much more useful for researchers. The integration with CIL opens the window towards application of advanced optimisation algorithms in PET/MR. 
Type Of Technology Software 
Year Produced 2019 
Open Source License? Yes  
Impact This version of the software allowed us to train a substantial of ECRs in PET/MR/CT reconstruction with highlight our training school in Chester, November 2019 with about 50 attendees, see https://www.ccppetmr.ac.uk/node/200 
URL http://www.ccppetmr.ac.uk/
 
Title STIR: Software for Tomographic Image Reconstruction 
Description Research Software for image reconstruction for Positron Emission Tomography (PET) and Single Photon Emission Tomography (SPECT). STIR is a long-running project (first release in 2001) with contributions from various universities and companies over the years. A major update has now been made available that uses OpenMP for multi-threading and has an interface to Python and MATLAB. 
Type Of Technology Software 
Year Produced 2015 
Open Source License? Yes  
Impact STIR is widely used as a research tool with a total of more than 260 citations in journals and conference proceedings. It has been used as the initial reconstruction software for the ClearPET preclinical PET scanner, allowing much faster introduction to the market. The scatter correction component of STIR has been used as tools for evaluation of a 3D method and convinced GE Healthcare to modify their own scatter correction software accordingly. The current update considerably enhances usability for research purposes. Previous versions of STIR was command-line based. Cluster-based parallelisation using MPI was already available in STIR. However, many researchers prefer to use interactive languages such that it is easier to try new ideas and get easier feedback via data examination. The multi-threading enhancement speeds-up computation time for interactive use. 
URL http://stir.sourceforge.net/
 
Description Hackathon 1 - 2018 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact A two-day Hackathon, in which developers met to tackle outstanding features. This allowed developers to exchange ideas and competencies, which benefitted new users in particular. Furthermore, the general public benefitted as this concentrated effort of code improvement increased the available features.
Year(s) Of Engagement Activity 2018
URL https://www.ccppetmr.ac.uk/hackathon1
 
Description Hackathon 2 - 2018 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Other audiences
Results and Impact A three-day Hackathon, in which developers met to tackle outstanding features. This allowed developers to exchange ideas and competencies, which benefitted new users in particular. Furthermore, the general public benefitted as this concentrated effort of code improvement increased the available features.

Particular effort was made in this Hackathon to include researchers that had not previously used the software. These efforts helped increase the user-base of the framework (benefitting the project), and allowed the new users to learn in a hands-on environment.
Year(s) Of Engagement Activity 2018
URL https://www.ccppetmr.ac.uk/node/162
 
Description Hackathon 3 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Other audiences
Results and Impact A three-day Hackathon, in which developers met to get experience with our software SIRF and various Machine Learning frameworks. This hackathon allowed participants to get hands-on experience with novel methods for using Machine Learning for image reconstruction. In addition, we developed extra functionality within our software SIRF to enable this functionality, including GPU computing capability.
Year(s) Of Engagement Activity 2019
URL http://www.ccppetmr.ac.uk/node/190
 
Description Hackathon 4 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Other audiences
Results and Impact This three day hackathon was a joint event between our CCP PETMR network and CCPi. Its aim was to prepare for a training school on synergistic image reconstruction. We added functionality to both SIRF and CIL and in particular prepared Jupyter notebooks with demos and exercises, lowering the entry-level for using our software, but also expanding its use for educational purposes.
Year(s) Of Engagement Activity 2019
URL http://www.ccppetmr.ac.uk/node/194
 
Description Hackathon 5 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact This three day hackathon involved researchers from UK and Germany and concentrated on adding motion correction capabilities to our SIRF software.
Year(s) Of Engagement Activity 2020
URL http://www.ccppetmr.ac.uk/node/233
 
Description Regular meetings for methods and software for synergistic image reconstruction 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact As part of our network, we hold regular meetings covering topical issues for synergistic reconstruction in PET/MR. They often have an educational component, some seminars, combined with active discussions. The meetings occurred roughly 6 weekly and have an average attendance of around 20 people, about 30% remotely of which about one third internationally.
Year(s) Of Engagement Activity 2015,2016,2017,2018,2019,2020
URL https://www.ccppetmr.ac.uk/node/228
 
Description STIR User's and Developer's Meeting 2018 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact 48 attendants, ranging from academic researchers, students and industry representatives attended a 2 hour workshop on our open source software STIR and SIRF. Our own group and external researchers presented their recent work with the software, with lots of interactions with the audience on capabilties, difficulties encountered, and how to access these contributions in the future.
Year(s) Of Engagement Activity 2018
URL http://stir.sourceforge.net/MIC2018UsersMeeting/
 
Description Seminar (Johan Nuyts): Reconstruction with MR-prior for PET brain imaging 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact Seminar given by external expert (Johan Nuyts - KUL, Belgium) on the subject of: Reconstruction with MR-prior for PET brain imaging.
Year(s) Of Engagement Activity 2018
 
Description Seminar (Simon Stute): CASToR - Customizable and Advanced Software for Tomographic Reconstruction 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Undergraduate students
Results and Impact Semiar given by external expert on alternative PET reconstruction toolkit. Converstation followed seminar on possibility of collaboration/incorporation of CASToR into our framework.
Year(s) Of Engagement Activity 2018
URL http://www.ccppetmr.ac.uk/node/153
 
Description Seminar (Simon Stute): PET reconstruction of the posterior image probability, including multimodal images 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Postgraduate students
Results and Impact Seminar by external expert on the subject: PET reconstruction of the posterior image probability, including multimodal images
Year(s) Of Engagement Activity 2018
URL http://www.ccppetmr.ac.uk/node/152
 
Description Symposium on Synergistic Image Reconstruction 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Traditionally, image reconstruction has focussed on estimating 2D or 3D images from a single modality data-set and acquisition. In recent years however, there have been significant developments in hardware and systems that allow extracting multi-parametric images from multiple data-sets. Examples include multi-spectral CT, multi-sequence MR, multi-modality such as PET-MR, and acquisitions from multiple-time points. While it is possible with such data to reconstruct several images independently, each corresponding to a different parameter or time-point, it is often advantageous, although challenging, to synergistically reconstruct all images.

This event brought a group of researchers together from these different fields and application areas, including medical and industrial imaging, to disseminate ideas to, and learn from, researchers both within and outside their usual research field.

This was a joint activity organised together with CCPi.
Year(s) Of Engagement Activity 2019
URL http://www.ccppetmr.ac.uk/symposium2019
 
Description Symposium on current technical challenges in clinical research using PET/MR 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact This symposium provided a platform for discussions on how to overcome the remaining barriers in adopting PET/MR in large clinical trials, while also looking at mechanisms for translation of research methods into practice. There was excellent interaction between speakers and audience (many of whom were international), including a discussion panel. Future collaborations were discussed.
Year(s) Of Engagement Activity 2020
URL http://www.ccppetmr.ac.uk/node/231
 
Description Training School in Synergistic Image Reconstruction 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact In this 2 day School we presented theory and practice of up-to-date synergistic reconstruction algorithms for PET, MR and (multi-spectral) CT. A special feature was the hands-on training using the SIRF and CIL software packages. The half day was dedicated for participants to tackle relevant research questions in groups.

The School was organised together with CCPi
Year(s) Of Engagement Activity 2019
URL https://www.ccppetmr.ac.uk/node/200
 
Description Training school on PET/MR reconstruction at PSMR 2018 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact We organised a one day PET-MRI School for students and early stage researchers at PSMR 2018, the 7th Conference on PET-MRI and SPECT-MRI in May 2018, Isle de Elba, Italy, (24 attendants) with a hands-on PET-MR software training session using SIRF. We funded UK attendants to the school.
Year(s) Of Engagement Activity 2018
URL https://www.ccppetmr.ac.uk/psmr2018
 
Description participation in the Siemens PET/MR User's Meeting 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact We participated in organising a User's Meeting together with Siemens Healthineers. The aim of the meeting was to give a status report of where PET/MR is in clinical practice, but also determine future directions. We also provided some information on our software or image reconstruction SIRF as an open source alternative for the manufacturer's dedicated platform.
Year(s) Of Engagement Activity 2020
 
Description seminar Dr Christoph Kolbitsch on Motion correction for cardiac PET-MR 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact Dr Kolbitsch and his PhD student Johannes Mayer gave a joint seminar on their recent work on enabling cardiac PET/MR imaging with motion correction for both respiration and cardiac contraction. The talk included information on clinical progress, but also a software framework, based on our SIRF software, for simulation of the methods, allowing validation with ground truth data.
Year(s) Of Engagement Activity 2020
URL http://www.ccppetmr.ac.uk/node/237