Dynamic High Resolution Photoacoustic Tomography System
Lead Research Organisation:
University College London
Department Name: Computer Science
Abstract
Biomedical imaging encompasses methods that measure almost every type of wave and particle including acoustic, electrical, optical and nuclear. Often there is a tradeoff between those systems that give high resolution structural images, but do not discriminate different physiological states well in terms of contrast, and those with good physiological contrast, but poor resolution. Photoacoustic imaging is an example of a "coupled Physics" system because it measures contrast in the optical part of the spectrum, which has high spectral sensitivity for different tissues, but uses sound to give high resolution. It works the same way as thunder is generated from a lightning strike, but on a very much smaller scale: a flash of light is shone onto a specimen and very small waves of sound are emitted when the light heats tissue a few fractions of a degree. We measure the sound with a very high resolution sensor array over space and time and use computer programs to recombine these measurements into 3D images. However, at present, this data takes several minutes to collect, so the imaging is limited to specimens that are static in time. In this proposal we aim to make this process hundreds of times faster, using a new mathematical sensing theory inspired by image compression - the technique that allows significant reduction in the size of an image file on the disk of a digital camera without visually diminishing the image quality.
The acoustic field on the sensor array as the photoacoustic wave passes through is a time-varying 2D function. This function, at a single moment in time, can be considered as the sum of basic patterns (rather like the way a time series can be decomposed into a number of frequency components). It turns out that frequently these basic patterns can be chosen so that there are relatively few of them which contribute to the function. However, as we do not know apriori which ones those are, we cannot measure their contribution directly. In this case the mathematical theory tells us that the best we can do is to sense the function using interrogation patterns which are as uncorrelated with the basic patterns as possible. If the number of basic patterns needed to accurately describe the field is small, then we only need relatively few of the interrogation patterns to capture the information in the acoustic wave. This is known as compressed sensing, and the challenge is to find such sets of the basic and interrogating patterns, that the number of measurements required to describe the field accurately is as small as possible.
Based on this idea, in this project we are going to build a photoacoustic system that measures the emitted sound waves using such interrogation patterns, and test that it accurately captures all the required information in the data. At the same time we are going to develop the mathematics that determines which basic and interrogation patterns are best. We will apply the system to test cases of moving and flowing objects where we know exactly what the changes are, and then to real preclinical problems looking at the flow of blood in the capillaries of small animals such as mice. This new system will enable us to look at the change in the oxygen consumption of the brain of animals which will tell us exactly which parts of the brain relate to different functions. This information can be used to develop a model of how drugs are taken up in tissues of the body, and how they are metabolised or washed out over time.
Success in this project will be a major breakthrough in biomedical imaging, allowing high resolution in space and time of directly important measures of tissue state. It will bring together advanced optical and acoustic measurement systems with novel mathematics and computer programming. It will open up a new range of applications of photoacoustics and provide a unique tool to medical and biological scientists investigating the physiology of living specimens.
The acoustic field on the sensor array as the photoacoustic wave passes through is a time-varying 2D function. This function, at a single moment in time, can be considered as the sum of basic patterns (rather like the way a time series can be decomposed into a number of frequency components). It turns out that frequently these basic patterns can be chosen so that there are relatively few of them which contribute to the function. However, as we do not know apriori which ones those are, we cannot measure their contribution directly. In this case the mathematical theory tells us that the best we can do is to sense the function using interrogation patterns which are as uncorrelated with the basic patterns as possible. If the number of basic patterns needed to accurately describe the field is small, then we only need relatively few of the interrogation patterns to capture the information in the acoustic wave. This is known as compressed sensing, and the challenge is to find such sets of the basic and interrogating patterns, that the number of measurements required to describe the field accurately is as small as possible.
Based on this idea, in this project we are going to build a photoacoustic system that measures the emitted sound waves using such interrogation patterns, and test that it accurately captures all the required information in the data. At the same time we are going to develop the mathematics that determines which basic and interrogation patterns are best. We will apply the system to test cases of moving and flowing objects where we know exactly what the changes are, and then to real preclinical problems looking at the flow of blood in the capillaries of small animals such as mice. This new system will enable us to look at the change in the oxygen consumption of the brain of animals which will tell us exactly which parts of the brain relate to different functions. This information can be used to develop a model of how drugs are taken up in tissues of the body, and how they are metabolised or washed out over time.
Success in this project will be a major breakthrough in biomedical imaging, allowing high resolution in space and time of directly important measures of tissue state. It will bring together advanced optical and acoustic measurement systems with novel mathematics and computer programming. It will open up a new range of applications of photoacoustics and provide a unique tool to medical and biological scientists investigating the physiology of living specimens.
Planned Impact
Photoacoustic tomography (PAT) is a recent and highly innovative technique that gives, in the form that we have developed at UCL, very high resolution 3D structural images of biomedical specimens. This project will provide a new high speed 4D (i.e. space-plus-time) imaging system for dynamic photoacoustic imaging. Impact will arise as a consequence of i) high frame rate, reducing image artefacts arising from patient/subject motion (an essential pre-requisite for clinical imaging); ii) ability to monitor dynamic anatomical and physiological events in real time. These innovations will in turn serve as enabling factors for in vivo implementation of a number of critically important photoacoustic imaging techniques.
For clinical imaging, the sytem will open up a broad range of clinical imaging applications that could not otherwise be realised, including the non-invasive diagnosis and staging of skin tumours and the assessment of superficial soft tissue damage associated with burns, wounds and ulcers, and to image skin perfusion following surgical procedures in order to assess tissue viability and predict outcome. It could also be applied to endoscopic imaging for assessment and treatment monitoring of coronary artery disease, prostate cancer and gastrointestinal pathologies. The path to clinical application is facilitated by the well established links between the UCL Department of Medical Physics and Bioengineering, UCL Centre for Medical Image Computing (CMIC) and clinicians at UCL Hospitals (UCLH) and within the UCL-Kings College London Comprehensive Cancer Imaging Centre (CCIC), which have previously led to the translation of a number of biomedical optical techniques to clinical application.
For preclinical imaging, development of a real time dynamic imaging capability will significantly extend the preclinical utility of the technology in oncology, cardiovascular medicine and neurology. As well as being able to image physiologically induced changes in blood flow and oxygenation as mentioned above it will enable visualisation of a range of dynamic processes such as vasomotion, tissue perfusion and drug pharmokinetics. Applications include basic research studies of oxygen supply and consumption, mapping blood oxygenation and flow in tumours to assess therapeutic response, functional imaging studies of the brain to help understand conditions such as stroke and epilepsy and the assessment of cardiac motion for studying heart function.
In the wider research community, imaging science is one of the fastest growing topics in mathematical science. We expect to initiate a new research field of "dynamic imaging from coupled Physics" which is likely to provoke intense interest from mathematicians, physicists, computational scientists, and engineers. We expect the techniques we develop to be taken up in several other applications both within biomedical imaging and further, in non-destructive testing, industrial tomography, and in geophysical applications including oil and gas exploration.
The reconstruction and image analysis software will be available as open-source for anyone to download. Experience tells us that such a mechanism makes an enormous difference to the impact of new computational methods, because it allows users to test and evaluate the techniques to an unlimited extent. Feedback in terms of bug reports, discussions, and contributed improvements can take place much faster than through formal publication channels, and will lead to strong evidence for confidence via extensive independent testing.
For clinical imaging, the sytem will open up a broad range of clinical imaging applications that could not otherwise be realised, including the non-invasive diagnosis and staging of skin tumours and the assessment of superficial soft tissue damage associated with burns, wounds and ulcers, and to image skin perfusion following surgical procedures in order to assess tissue viability and predict outcome. It could also be applied to endoscopic imaging for assessment and treatment monitoring of coronary artery disease, prostate cancer and gastrointestinal pathologies. The path to clinical application is facilitated by the well established links between the UCL Department of Medical Physics and Bioengineering, UCL Centre for Medical Image Computing (CMIC) and clinicians at UCL Hospitals (UCLH) and within the UCL-Kings College London Comprehensive Cancer Imaging Centre (CCIC), which have previously led to the translation of a number of biomedical optical techniques to clinical application.
For preclinical imaging, development of a real time dynamic imaging capability will significantly extend the preclinical utility of the technology in oncology, cardiovascular medicine and neurology. As well as being able to image physiologically induced changes in blood flow and oxygenation as mentioned above it will enable visualisation of a range of dynamic processes such as vasomotion, tissue perfusion and drug pharmokinetics. Applications include basic research studies of oxygen supply and consumption, mapping blood oxygenation and flow in tumours to assess therapeutic response, functional imaging studies of the brain to help understand conditions such as stroke and epilepsy and the assessment of cardiac motion for studying heart function.
In the wider research community, imaging science is one of the fastest growing topics in mathematical science. We expect to initiate a new research field of "dynamic imaging from coupled Physics" which is likely to provoke intense interest from mathematicians, physicists, computational scientists, and engineers. We expect the techniques we develop to be taken up in several other applications both within biomedical imaging and further, in non-destructive testing, industrial tomography, and in geophysical applications including oil and gas exploration.
The reconstruction and image analysis software will be available as open-source for anyone to download. Experience tells us that such a mechanism makes an enormous difference to the impact of new computational methods, because it allows users to test and evaluate the techniques to an unlimited extent. Feedback in terms of bug reports, discussions, and contributed improvements can take place much faster than through formal publication channels, and will lead to strong evidence for confidence via extensive independent testing.
Organisations
Publications
Ringholm T
(2018)
Variational Image Regularization with Euler's Elastica Using a Discrete Gradient Scheme
in SIAM Journal on Imaging Sciences
Ringholm T
(2018)
Variational Image Regularization with Euler's Elastica Using a Discrete Gradient Scheme
in SIAM Journal on Imaging Sciences
Calatroni L
(2018)
Unveiling the invisible: mathematical methods for restoring and interpreting illuminated manuscripts.
in Heritage science
Lucka, F
(2015)
Towards 4D Photoacoustic Tomography
Tick J
(2018)
Three dimensional photoacoustic tomography in Bayesian framework.
in The Journal of the Acoustical Society of America
Adler J
(2018)
Task adapted reconstruction for inverse problems
Adler J
(2022)
Task adapted reconstruction for inverse problems
in Inverse Problems
Huynh N
(2016)
Single-pixel optical camera for video rate ultrasonic imaging
in Optica
Description | We have confirmed our hypothesis that PAT images can be reconstructed from undersampled data without sacrificing image quality, by using techniques from compressed sensing. We developed novel instrumentation and algorithms for achieving close-to-realtime high-resolution 3D dynamic images. Compression in acquisition was achieved using Instrumentation based on i) digital micro-mirror devices ii) steerable parallel beam readout. Algorithms were developed using: i) reconstruction in data space of optically compressed data from the DMD system followed by time-reversal reconstruction. Advantage : highly parallelisable, agnostic with respect to details of forward computational model ii) 3D reconstruction of a PAT image using optimised 3D curvelet representation in image space, mapped to novel band-limited 2D curvelet representation in data space. Advantage : higher compression rates iii) Dynamic reconstruction using time-varying spatial sampling and exploiting a spatial-temporal optical flow constraint in reconstruction. Advantage : maximal 4D compression rates. iv) Machine learning approaches based on deep gradient descent improved reconstruction speed and accuracy using separate networks per iteration trained on a databased of vascular lung images. v) Using a fast approximate forward model had comparable quality of image reconstruction at a fraction of the computational cost. |
Exploitation Route | The methods of spatio-temporal postprocessing and visualisation of 4D PAT data, developed are not restricted to PAT and can be potentially used in any application with dynamic 4D-data. The methods based on deep gradient descent can also be widely applied in other modalities such as CT and MRI. Use of fast approximate models allows the use of learning based reconstruction methods in problems of much higher dimension such as high resolution X-Ray tomography. |
Sectors | Healthcare |
Description | We have managed to reduce acquisition time from several minutes to a few seconds. This has opened the door to commercial development of photoacoustic imaging. The patent awarded has been licensed to the UCL Spinout company DeepColor The methods developed for accelerated image acquisition led to a collaboration with LightPoint Medical to incorporate Cherenkov Light Imaging into Peri-Operative Robotic Prostate Surgery. Methods combining deep learning and model based reconstruction have become an increasingly prevalent topic in both machine learning and inverse problem communities. It is increasingly recognised that neither a full data driven nor fully Physics driven analysis is sufficient to fully realise the potential of novel imaging modalities like photoacoustics. Several conferences and special issues of journals are being devoted to this topic. |
First Year Of Impact | 2016 |
Sector | Healthcare |
Impact Types | Economic |
Description | EPSRC Translational Alliance Partnership Scheme |
Amount | £242,828 (GBP) |
Funding ID | EP/N022750/1 |
Organisation | Engineering and Physical Sciences Research Council (EPSRC) |
Sector | Public |
Country | United Kingdom |
Start | 05/2016 |
End | 05/2018 |
Description | Endoscopic photoacoustic devices for minimally invasive biomedical sensing and imaging |
Amount | £628,525 (GBP) |
Funding ID | EP/L002019/1 |
Organisation | Engineering and Physical Sciences Research Council (EPSRC) |
Sector | Public |
Country | United Kingdom |
Start | 01/2014 |
End | 07/2017 |
Description | External Partnership Funding |
Amount | £4,966,856 (GBP) |
Funding ID | NS/A000027/1 |
Organisation | Engineering and Physical Sciences Research Council (EPSRC) |
Sector | Public |
Country | United Kingdom |
Start | 03/2014 |
End | 03/2021 |
Title | APPARATUS AND METHOD FOR PERFORMING PHOTOACOUSTIC TOMOGRAPHY |
Description | A method and apparatus are provided for performing photoacoustic tomography with respect to a sample that receives a pulse of excitation electromagnetic radiation and generates an acoustic field in response to said pulse. One embodiment provides an apparatus comprising an acoustically sensitive surface, wherein the acoustic field generated in response to said pulse is incident upon said acoustically sensitive surface to form a signal. The apparatus further comprises a source for directing an interrogation beam of electromagnetic radiation onto said acoustically sensitive surface so as to be modulated by the signal; means for applying a sensitivity pattern to the interrogation beam; and a read-out system for receiving the interrogation beam from the acoustically sensitive surface and for determining a value representing a spatial integral of the signal across the acoustically sensitive surface, wherein said spatial integral is weighted by the applied sensitivity pattern. The apparatus is configured to apply a sequence of sensitivity patterns to the interrogation beam and to determine a respective sequence of values for said weighted spatial integral for generating a photoacoustic image. |
IP Reference | WO2014207440 |
Protection | Patent granted |
Year Protection Granted | 2014 |
Licensed | Yes |
Impact | This patent has been used as a component for an EPSRC-Wellcome grant WT101957 "Image-guided Intrauterine Minimally Invasive Fetal Diagnosis and Therapy" It has been licensed to the "DeepColor" Spinout from UCL |
Company Name | DEEPCOLOR |
Description | UCL spinout that has licensed IP generated under a range of EPSRC funded projects. The aim is to develop medical photoacoustic imaging scanners for the diagnosis and treatment monitoring of cancer and other diseases |
Year Established | 2016 |
Impact | The company has recently obtained seed funding from angel investors. |