Bayesian computation for low-photon imaging
Lead Research Organisation:
Heriot-Watt University
Department Name: S of Mathematical and Computer Sciences
Abstract
Images are rich in data of significant economic and social value, and over the past decade, they have become fundamental sources of information in many disciplines (e.g., medicine, biology, agriculture, defence, earth sciences, and non-destructive testing). These disciplines now drive the development of sophisticated and specialised imaging devices. Such devices tightly combine two forms of innovation to deliver state-of-the-art performance: 1) sophisticated instrumentation and sensors that push technology and physics to the limits, and 2) highly advanced computational imaging (CI) methods that carefully analyse the generated raw data to produce sharp images with fine detail.
This proposal focuses on CI methodology for quantum-enhanced imaging, a new imaging paradigm that seeks to exploit the quantum nature of light to go far beyond what is possible in classical optics in terms of spatial and temporal resolution and dynamic range. This transformative approach is poised to dramatically advance imaging technologies and generate great social and economic impact. To make sure that the UK is at the forefront of this strategic technological developments, the UK government created the Quantum Enhanced Imaging Hub (QUANTIC) in 2014 as part of the UK National Quantum Technology Programme, which was renewed this year. QUANTIC has developed impressive new sensors for extreme imaging conditions. However, these advances in sensor technology have not been matched by progress in CI methodology, gravely jeopardizing the impact of these promising technologies.
The aim of this proposal is to develop CI methodology specifically designed for solving quantum-enhanced imaging problems in which very few photons are observed (i.e., low-photon and single-photon imaging problems). Our methods will be formulated in the Bayesian statistical framework, which is particularly appropriate for solving these challenging imaging problems because: 1) it enables the use of sophisticated statistical models to accurately describe the underlying physics, 2) it allows the automatic calibration of models, and 3) it provides tools to quantify the uncertainty in the solutions delivered.
At present, the benefits and superior performance of Bayesian statistical CI methods is obtained at the expense of a prohibitively high computational cost. We plan to significantly accelerate Bayesian solutions for quantum-enhanced imaging problems by developing specialised computation methods that combine and extend ideas from different areas of applied mathematics, computational statistics, and artificial intelligence.
We believe that the availability of fast Bayesian computation methods will unlock the potential of these promising quantum-enhanced imaging technologies and lead to their wide adoption in science and engineering, generating generate great social and economic benefit through an impact on medicine, biology, agriculture, defence, earth sciences, and non-destructive testing.
In order to guarantee this impact, during the project, we will apply the proposed methods to three important quantum-enhanced imaging problems (low-photon multispectral single-pixel imaging, high-resolution PGET, and single-photon 3D LIDAR with array sensors). These applications will be investigated in collaboration with world-leading experts who will provide data and training, and help disseminate the research outputs. To maximise the impact of our work, we will also develop open-source software - with documentation and demonstrations - that we will share online and use in outreach activities aimed at informing the public about STEM research and inspiring young people to pursue STEM careers. This project will also help train the next generation of top-tier talent in AI and quantum technology.
This proposal focuses on CI methodology for quantum-enhanced imaging, a new imaging paradigm that seeks to exploit the quantum nature of light to go far beyond what is possible in classical optics in terms of spatial and temporal resolution and dynamic range. This transformative approach is poised to dramatically advance imaging technologies and generate great social and economic impact. To make sure that the UK is at the forefront of this strategic technological developments, the UK government created the Quantum Enhanced Imaging Hub (QUANTIC) in 2014 as part of the UK National Quantum Technology Programme, which was renewed this year. QUANTIC has developed impressive new sensors for extreme imaging conditions. However, these advances in sensor technology have not been matched by progress in CI methodology, gravely jeopardizing the impact of these promising technologies.
The aim of this proposal is to develop CI methodology specifically designed for solving quantum-enhanced imaging problems in which very few photons are observed (i.e., low-photon and single-photon imaging problems). Our methods will be formulated in the Bayesian statistical framework, which is particularly appropriate for solving these challenging imaging problems because: 1) it enables the use of sophisticated statistical models to accurately describe the underlying physics, 2) it allows the automatic calibration of models, and 3) it provides tools to quantify the uncertainty in the solutions delivered.
At present, the benefits and superior performance of Bayesian statistical CI methods is obtained at the expense of a prohibitively high computational cost. We plan to significantly accelerate Bayesian solutions for quantum-enhanced imaging problems by developing specialised computation methods that combine and extend ideas from different areas of applied mathematics, computational statistics, and artificial intelligence.
We believe that the availability of fast Bayesian computation methods will unlock the potential of these promising quantum-enhanced imaging technologies and lead to their wide adoption in science and engineering, generating generate great social and economic benefit through an impact on medicine, biology, agriculture, defence, earth sciences, and non-destructive testing.
In order to guarantee this impact, during the project, we will apply the proposed methods to three important quantum-enhanced imaging problems (low-photon multispectral single-pixel imaging, high-resolution PGET, and single-photon 3D LIDAR with array sensors). These applications will be investigated in collaboration with world-leading experts who will provide data and training, and help disseminate the research outputs. To maximise the impact of our work, we will also develop open-source software - with documentation and demonstrations - that we will share online and use in outreach activities aimed at informing the public about STEM research and inspiring young people to pursue STEM careers. This project will also help train the next generation of top-tier talent in AI and quantum technology.
Planned Impact
Images are fundamental sources of information in many disciplines, such as medicine, biology, agriculture, defence, earth sciences, and non-destructive testing. These disciplines now drive the development of sophisticated and specialised imaging devices. Such devices tightly combine two forms of innovation to deliver state-of-the-art performance: 1) sophisticated instrumentation and sensors that push technology and physics to the limits; and 2) advanced computational imaging (CI) methods that carefully analyse the generated raw data to produce sharp images with fine detail.
This proposal focuses on CI methodology for quantum-enhanced imaging problem. Quantum-enhanced imaging is a new imaging paradigm that seeks to exploit the quantum nature of light to go far beyond what is possible in classical optics in terms of spatial and temporal resolution and dynamic range. This transformative approach is poised to dramatically advance imaging technologies and generate great social and economic impact. Maximising the impact of these technologies is an important aspect of the UK National Quantum Technology Programme.
Important recent investments have led to the development of impressive new instrumentation and sensors for quantum-enhanced imaging in extreme imaging conditions. However, these advances in sensor technology have not been matched by progress in CI methodology, gravely jeopardizing the impact of these promising technologies. In this proposal, we aim to develop the new generation of fast and powerful CI methods specifically designed for quantum-enhanced imaging problems involving photon-starved measurements. These CI methods will unlock the potential of these promising quantum-enhanced imaging technologies and lead to their wide adoption in medicine, biology, biology, agriculture, defence, earth sciences, and non-destructive testing.
In order to guarantee this impact, during the project, the proposed methods will be applied to three important quantum-enhanced imaging problems: low-photon multispectral single-pixel imaging, high-resolution passive gamma emission tomography, and single-photon 3D LIDAR with array sensors. These applications will be investigated in collaboration with world-leading experts who will provide data and training, and help disseminate the research outputs. To further amplify the impact of our work, we will also develop software tools for the considered applications, which we will design jointly with our collaborators to ensure that they meet up the expectations of the end-users. We will also make publicly available open-source implementations of our methods with documentation and demonstrations to help other mathematicians and imaging scientists engage with the UK National Quantum Technology agenda.
The team of researchers involved in this project will participate in several public engagement events aimed at inspiring and informing the public about STEM research. Examples include participating in science festivals to talk about AI&Data and visiting schools in Edinburgh to meet students in Year 9 to encourage them to pursue STEM subjects. The talks and visits will involve a mix of demonstrations, interactive activities, and competitions, all related to STEM through the prism of imaging sciences.
This proposal focuses on CI methodology for quantum-enhanced imaging problem. Quantum-enhanced imaging is a new imaging paradigm that seeks to exploit the quantum nature of light to go far beyond what is possible in classical optics in terms of spatial and temporal resolution and dynamic range. This transformative approach is poised to dramatically advance imaging technologies and generate great social and economic impact. Maximising the impact of these technologies is an important aspect of the UK National Quantum Technology Programme.
Important recent investments have led to the development of impressive new instrumentation and sensors for quantum-enhanced imaging in extreme imaging conditions. However, these advances in sensor technology have not been matched by progress in CI methodology, gravely jeopardizing the impact of these promising technologies. In this proposal, we aim to develop the new generation of fast and powerful CI methods specifically designed for quantum-enhanced imaging problems involving photon-starved measurements. These CI methods will unlock the potential of these promising quantum-enhanced imaging technologies and lead to their wide adoption in medicine, biology, biology, agriculture, defence, earth sciences, and non-destructive testing.
In order to guarantee this impact, during the project, the proposed methods will be applied to three important quantum-enhanced imaging problems: low-photon multispectral single-pixel imaging, high-resolution passive gamma emission tomography, and single-photon 3D LIDAR with array sensors. These applications will be investigated in collaboration with world-leading experts who will provide data and training, and help disseminate the research outputs. To further amplify the impact of our work, we will also develop software tools for the considered applications, which we will design jointly with our collaborators to ensure that they meet up the expectations of the end-users. We will also make publicly available open-source implementations of our methods with documentation and demonstrations to help other mathematicians and imaging scientists engage with the UK National Quantum Technology agenda.
The team of researchers involved in this project will participate in several public engagement events aimed at inspiring and informing the public about STEM research. Examples include participating in science festivals to talk about AI&Data and visiting schools in Edinburgh to meet students in Year 9 to encourage them to pursue STEM subjects. The talks and visits will involve a mix of demonstrations, interactive activities, and competitions, all related to STEM through the prism of imaging sciences.
Organisations
- Heriot-Watt University (Lead Research Organisation)
- Charles III University of Madrid (Collaboration)
- Alternative Energies and Atomic Energy Commission (CEA) (Collaboration)
- École normale supérieure de Lyon (ENS Lyon) (Collaboration)
- University of Paris - Descartes (Collaboration)
- University of Illinois Urbana-Champaign (Collaboration, Project Partner)
- École Normale Supérieure - PSL (Project Partner)
Publications
Li Z
(2022)
Sparse Linear Spectral Unmixing of Hyperspectral Images Using Expectation-Propagation
in IEEE Transactions on Geoscience and Remote Sensing
Melidonis S
(2022)
Efficient Bayesian computation for low-photon imaging problems
Melidonis S
(2023)
Efficient Bayesian Computation for Low-Photon Imaging Problems
in SIAM Journal on Imaging Sciences
Melidonis S
(2024)
Empirical Bayesian Imaging With Large-Scale Push-Forward Generative Priors
in IEEE Signal Processing Letters
Melidonis S
(2022)
Efficient Bayesian computation for low-photon imaging problems
Description | Modern imaging and computer vision systems are increasingly required to operate in extreme conditions (\eg, ultra-fast acquisition times, low illumination, long-range, unconventional environments). This has led to the development of new quantum-enhanced sensors and cameras that exploit the particle nature of light to exceed the limitations of classical imaging strategies. Unfortunately, the raw images produced by these new cameras are of very poor quality. We have developed new mathematics and algorithms to significantly enhance the quality of these images. Following some successful preliminary experiments, we are currently investigating the application of this new mathematics and algorithms to real data. |
Exploitation Route | We anticipate that future research will specialise the proposed algorithms for specific applications of quantum-enhanced imaging, and subsequently integrate them within new imaging systems and software. |
Sectors | Aerospace Defence and Marine Agriculture Food and Drink Chemicals Energy Environment Healthcare |
Title | SR-SOUL |
Description | The super-resolution stochastic optimisation unadjusted Langevin algorithm is a Markov chain Monte Carlo sampler designed for performing Bayesian inference with highly realistic image priors that are represented by a generative model trained for image super-resolution. This model is typically encoded by a deep neural network that has been trained with application-specific data, for example a normalising flow. |
Type Of Material | Computer model/algorithm |
Year Produced | 2023 |
Provided To Others? | Yes |
Impact | No impact yet. |
URL | https://researchportal.hw.ac.uk/en/publications/empirical-bayesian-imaging-with-large-scale-push-for... |
Description | Dr. Angela Di Fulvio |
Organisation | University of Illinois |
Country | United States |
Sector | Academic/University |
PI Contribution | We have led the development of the new Bayesian AI techniques for a new Compton-based imager developed at the University of Illinois. |
Collaborator Contribution | Angela Di Fulvio and her group have designed the imager and shared simulated and real data to support the development of the Bayesian AI technique, as well as provided feedback on the results delivered by the technique. |
Impact | This is a multi-disciplinary collaboration at the interface of imaging sciences, nuclear sciences and mathematical statistics. Research paper currently under consideration for publication. |
Start Year | 2019 |
Description | Julian Tachella (ENS Lyon) |
Organisation | École normale supérieure de Lyon (ENS Lyon) |
Country | France |
Sector | Academic/University |
PI Contribution | We have co-created a new unsupervised statistical machine learning method that is able to quantify the uncertainty in the solution of an imaging problem directly from measurements, by leveraging symmetries and invariances in the problem, without the need for ground truth data. |
Collaborator Contribution | Co-creation of the method and implementation of algorithms and experiments |
Impact | https://arxiv.org/pdf/2310.11838.pdf (accepted for publication in AISTATS 2024). |
Start Year | 2022 |
Description | Julie Delon and Andres Almansa |
Organisation | University of Paris - Descartes |
Country | France |
Sector | Academic/University |
PI Contribution | We have led the development of the new Bayesian AI techniques for imaging problems where the prior knowledge is available in the form of a set of training examples. |
Collaborator Contribution | Julie Delon and Andres Almansa have led the machine learning aspects of the work and the algorithmic implementation of the proposed techniques. |
Impact | This is a multi-disciplinary collaboration at the interface of imaging sciences, applied analysis, machine learning, probability theory and mathematical statistics. |
Start Year | 2019 |
Description | Prof. Jesus Maria Sanz Serna |
Organisation | Charles III University of Madrid |
Country | Spain |
Sector | Academic/University |
PI Contribution | We have been analysing the convergence properties of unconstrained and constrained convex optimisation algorithms for low-photon imaging problems. |
Collaborator Contribution | Prof. Jesus Maria Sanz Serna has contributed technical expertise related to the analysis of the convergence rate. |
Impact | Research manuscript currently in consideration for publication. |
Start Year | 2019 |
Description | Tobias Liaudiat (CEA Paris-Saclay) |
Organisation | Alternative Energies and Atomic Energy Commission (CEA) |
Country | France |
Sector | Public |
PI Contribution | We are developing a Bayesian computation method to detect when state-of-the-art generative image models have become unreliable for scientific imaging because of distribution shift, which we plan to apply to low-photon imaging problems. |
Collaborator Contribution | Dr Liaudiat is leading the implementation and experimental aspects of the project. |
Impact | No outputs yet. |
Start Year | 2023 |
Title | Equivariant bootstrapping for UQ in Imaging |
Description | The equivariant bootstrap algorithm is a scalable and intuitive resampling procedure which produces well-calibrated uncertainty regions for linear imaging problem and any reconstruction network (i.e., trained in a supervised or self-supervised manner). The method outperforms other state-of-the-art strategies in terms of accuracy and computational speed. |
Type Of Technology | Software |
Year Produced | 2024 |
Open Source License? | Yes |
Impact | No impact yet |
URL | https://arxiv.org/pdf/2310.11838.pdf |
Title | MCMC for Low Photon Imaging |
Description | A PyTorch code for a Markov chain Monte Carlo methodology to perform Bayesian inference in low-photon imaging problems, with particular attention given to situations involving observation noise processes that deviate significantly from Gaussian noise, such as binomial, geometric, and low-intensity Poisson noise. The methodology is based on a reflected and regularized Langevin SDE. The code includes demos related to image deblurring, denoising, and inpainting under binomial, geometric, and Poisson noise. |
Type Of Technology | Software |
Year Produced | 2023 |
Open Source License? | Yes |
Impact | No impact, yet. |
URL | https://epubs.siam.org/doi/10.1137/22M1502240 |
Description | Stakeholder 3-day workshop |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | The 3-day workshop Interfacing Bayesian Statistics, Machine Learning, Applied Analysis, and Blind and Semi-Blind Imaging Inverse Problems" focused on the Bayesian statistics, machine learning, and applied analysis frameworks for imaging inverse problems that are blind or semi-blind. The event brought together industrialists, students, world-leading experts and rising early career researchers to discuss recent developments in the fields as well as open challenges, with a focus on co-creation and on fostering synergies to pursue ideas that develop at the fertile interface where the three frameworks meet. |
Year(s) Of Engagement Activity | 2023 |
URL | https://www.icms.org.uk/workshops/2023/interfacing-bayesian-statistics-machine-learning-applied-anal... |