Bayesian computation for low-photon imaging

Lead Research Organisation: Heriot-Watt University
Department Name: S of Mathematical and Computer Sciences

Abstract

Images are rich in data of significant economic and social value, and over the past decade, they have become fundamental sources of information in many disciplines (e.g., medicine, biology, agriculture, defence, earth sciences, and non-destructive testing). These disciplines now drive the development of sophisticated and specialised imaging devices. Such devices tightly combine two forms of innovation to deliver state-of-the-art performance: 1) sophisticated instrumentation and sensors that push technology and physics to the limits, and 2) highly advanced computational imaging (CI) methods that carefully analyse the generated raw data to produce sharp images with fine detail.

This proposal focuses on CI methodology for quantum-enhanced imaging, a new imaging paradigm that seeks to exploit the quantum nature of light to go far beyond what is possible in classical optics in terms of spatial and temporal resolution and dynamic range. This transformative approach is poised to dramatically advance imaging technologies and generate great social and economic impact. To make sure that the UK is at the forefront of this strategic technological developments, the UK government created the Quantum Enhanced Imaging Hub (QUANTIC) in 2014 as part of the UK National Quantum Technology Programme, which was renewed this year. QUANTIC has developed impressive new sensors for extreme imaging conditions. However, these advances in sensor technology have not been matched by progress in CI methodology, gravely jeopardizing the impact of these promising technologies.

The aim of this proposal is to develop CI methodology specifically designed for solving quantum-enhanced imaging problems in which very few photons are observed (i.e., low-photon and single-photon imaging problems). Our methods will be formulated in the Bayesian statistical framework, which is particularly appropriate for solving these challenging imaging problems because: 1) it enables the use of sophisticated statistical models to accurately describe the underlying physics, 2) it allows the automatic calibration of models, and 3) it provides tools to quantify the uncertainty in the solutions delivered.

At present, the benefits and superior performance of Bayesian statistical CI methods is obtained at the expense of a prohibitively high computational cost. We plan to significantly accelerate Bayesian solutions for quantum-enhanced imaging problems by developing specialised computation methods that combine and extend ideas from different areas of applied mathematics, computational statistics, and artificial intelligence.

We believe that the availability of fast Bayesian computation methods will unlock the potential of these promising quantum-enhanced imaging technologies and lead to their wide adoption in science and engineering, generating generate great social and economic benefit through an impact on medicine, biology, agriculture, defence, earth sciences, and non-destructive testing.

In order to guarantee this impact, during the project, we will apply the proposed methods to three important quantum-enhanced imaging problems (low-photon multispectral single-pixel imaging, high-resolution PGET, and single-photon 3D LIDAR with array sensors). These applications will be investigated in collaboration with world-leading experts who will provide data and training, and help disseminate the research outputs. To maximise the impact of our work, we will also develop open-source software - with documentation and demonstrations - that we will share online and use in outreach activities aimed at informing the public about STEM research and inspiring young people to pursue STEM careers. This project will also help train the next generation of top-tier talent in AI and quantum technology.

Planned Impact

Images are fundamental sources of information in many disciplines, such as medicine, biology, agriculture, defence, earth sciences, and non-destructive testing. These disciplines now drive the development of sophisticated and specialised imaging devices. Such devices tightly combine two forms of innovation to deliver state-of-the-art performance: 1) sophisticated instrumentation and sensors that push technology and physics to the limits; and 2) advanced computational imaging (CI) methods that carefully analyse the generated raw data to produce sharp images with fine detail.

This proposal focuses on CI methodology for quantum-enhanced imaging problem. Quantum-enhanced imaging is a new imaging paradigm that seeks to exploit the quantum nature of light to go far beyond what is possible in classical optics in terms of spatial and temporal resolution and dynamic range. This transformative approach is poised to dramatically advance imaging technologies and generate great social and economic impact. Maximising the impact of these technologies is an important aspect of the UK National Quantum Technology Programme.

Important recent investments have led to the development of impressive new instrumentation and sensors for quantum-enhanced imaging in extreme imaging conditions. However, these advances in sensor technology have not been matched by progress in CI methodology, gravely jeopardizing the impact of these promising technologies. In this proposal, we aim to develop the new generation of fast and powerful CI methods specifically designed for quantum-enhanced imaging problems involving photon-starved measurements. These CI methods will unlock the potential of these promising quantum-enhanced imaging technologies and lead to their wide adoption in medicine, biology, biology, agriculture, defence, earth sciences, and non-destructive testing.

In order to guarantee this impact, during the project, the proposed methods will be applied to three important quantum-enhanced imaging problems: low-photon multispectral single-pixel imaging, high-resolution passive gamma emission tomography, and single-photon 3D LIDAR with array sensors. These applications will be investigated in collaboration with world-leading experts who will provide data and training, and help disseminate the research outputs. To further amplify the impact of our work, we will also develop software tools for the considered applications, which we will design jointly with our collaborators to ensure that they meet up the expectations of the end-users. We will also make publicly available open-source implementations of our methods with documentation and demonstrations to help other mathematicians and imaging scientists engage with the UK National Quantum Technology agenda.

The team of researchers involved in this project will participate in several public engagement events aimed at inspiring and informing the public about STEM research. Examples include participating in science festivals to talk about AI&Data and visiting schools in Edinburgh to meet students in Year 9 to encourage them to pursue STEM subjects. The talks and visits will involve a mix of demonstrations, interactive activities, and competitions, all related to STEM through the prism of imaging sciences.
 
Description Modern imaging and computer vision systems are increasingly required to operate in extreme conditions (\eg, ultra-fast acquisition times, low illumination, long-range, unconventional environments). This has led to the development of new quantum-enhanced sensors and cameras that exploit the particle nature of light to exceed the limitations of classical imaging strategies. Unfortunately, the raw images produced by these new cameras are of very poor quality. We have developed new mathematics and algorithms to significantly enhance the quality of these images. Following some successful preliminary experiments, we are currently investigating the application of this new mathematics and algorithms to real data.
Exploitation Route We anticipate that future research will specialise the proposed algorithms for specific applications of quantum-enhanced imaging, and subsequently integrate them within new imaging systems and software.
Sectors Aerospace

Defence and Marine

Agriculture

Food and Drink

Chemicals

Energy

Environment

Healthcare

 
Title SR-SOUL 
Description The super-resolution stochastic optimisation unadjusted Langevin algorithm is a Markov chain Monte Carlo sampler designed for performing Bayesian inference with highly realistic image priors that are represented by a generative model trained for image super-resolution. This model is typically encoded by a deep neural network that has been trained with application-specific data, for example a normalising flow. 
Type Of Material Computer model/algorithm 
Year Produced 2023 
Provided To Others? Yes  
Impact No impact yet. 
URL https://researchportal.hw.ac.uk/en/publications/empirical-bayesian-imaging-with-large-scale-push-for...
 
Description Dr. Angela Di Fulvio 
Organisation University of Illinois
Country United States 
Sector Academic/University 
PI Contribution We have led the development of the new Bayesian AI techniques for a new Compton-based imager developed at the University of Illinois.
Collaborator Contribution Angela Di Fulvio and her group have designed the imager and shared simulated and real data to support the development of the Bayesian AI technique, as well as provided feedback on the results delivered by the technique.
Impact This is a multi-disciplinary collaboration at the interface of imaging sciences, nuclear sciences and mathematical statistics. Research paper currently under consideration for publication.
Start Year 2019
 
Description Julian Tachella (ENS Lyon) 
Organisation École normale supérieure de Lyon (ENS Lyon)
Country France 
Sector Academic/University 
PI Contribution We have co-created a new unsupervised statistical machine learning method that is able to quantify the uncertainty in the solution of an imaging problem directly from measurements, by leveraging symmetries and invariances in the problem, without the need for ground truth data.
Collaborator Contribution Co-creation of the method and implementation of algorithms and experiments
Impact https://arxiv.org/pdf/2310.11838.pdf (accepted for publication in AISTATS 2024).
Start Year 2022
 
Description Julie Delon and Andres Almansa 
Organisation University of Paris - Descartes
Country France 
Sector Academic/University 
PI Contribution We have led the development of the new Bayesian AI techniques for imaging problems where the prior knowledge is available in the form of a set of training examples.
Collaborator Contribution Julie Delon and Andres Almansa have led the machine learning aspects of the work and the algorithmic implementation of the proposed techniques.
Impact This is a multi-disciplinary collaboration at the interface of imaging sciences, applied analysis, machine learning, probability theory and mathematical statistics.
Start Year 2019
 
Description Prof. Jesus Maria Sanz Serna 
Organisation Charles III University of Madrid
Country Spain 
Sector Academic/University 
PI Contribution We have been analysing the convergence properties of unconstrained and constrained convex optimisation algorithms for low-photon imaging problems.
Collaborator Contribution Prof. Jesus Maria Sanz Serna has contributed technical expertise related to the analysis of the convergence rate.
Impact Research manuscript currently in consideration for publication.
Start Year 2019
 
Description Tobias Liaudiat (CEA Paris-Saclay) 
Organisation Alternative Energies and Atomic Energy Commission (CEA)
Country France 
Sector Public 
PI Contribution We are developing a Bayesian computation method to detect when state-of-the-art generative image models have become unreliable for scientific imaging because of distribution shift, which we plan to apply to low-photon imaging problems.
Collaborator Contribution Dr Liaudiat is leading the implementation and experimental aspects of the project.
Impact No outputs yet.
Start Year 2023
 
Title Equivariant bootstrapping for UQ in Imaging 
Description The equivariant bootstrap algorithm is a scalable and intuitive resampling procedure which produces well-calibrated uncertainty regions for linear imaging problem and any reconstruction network (i.e., trained in a supervised or self-supervised manner). The method outperforms other state-of-the-art strategies in terms of accuracy and computational speed. 
Type Of Technology Software 
Year Produced 2024 
Open Source License? Yes  
Impact No impact yet 
URL https://arxiv.org/pdf/2310.11838.pdf
 
Title MCMC for Low Photon Imaging 
Description A PyTorch code for a Markov chain Monte Carlo methodology to perform Bayesian inference in low-photon imaging problems, with particular attention given to situations involving observation noise processes that deviate significantly from Gaussian noise, such as binomial, geometric, and low-intensity Poisson noise. The methodology is based on a reflected and regularized Langevin SDE. The code includes demos related to image deblurring, denoising, and inpainting under binomial, geometric, and Poisson noise. 
Type Of Technology Software 
Year Produced 2023 
Open Source License? Yes  
Impact No impact, yet. 
URL https://epubs.siam.org/doi/10.1137/22M1502240
 
Description Stakeholder 3-day workshop 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact The 3-day workshop Interfacing Bayesian Statistics, Machine Learning, Applied Analysis, and Blind and Semi-Blind Imaging Inverse Problems" focused on the Bayesian statistics, machine learning, and applied analysis frameworks for imaging inverse problems that are blind or semi-blind. The event brought together industrialists, students, world-leading experts and rising early career researchers to discuss recent developments in the fields as well as open challenges, with a focus on co-creation and on fostering synergies to pursue ideas that develop at the fertile interface where the three frameworks meet.
Year(s) Of Engagement Activity 2023
URL https://www.icms.org.uk/workshops/2023/interfacing-bayesian-statistics-machine-learning-applied-anal...