Sparse & Higher Order Image Restoration
Lead Research Organisation:
University of Cambridge
Department Name: Applied Maths and Theoretical Physics
Abstract
In the modern society we encounter digital images in many different situations: from everyday life, where analogue cameras have long been replaced by digital ones, to their professional use in medicine, earth sciences, arts, and security applications. Examples of medical imaging tools are MRI (Magnetic Resonance Imaging), PET (Positron Emission Tomography), CT (computed tomography) for imaging the brain and inner organs like the human heart. These imaging tools usually produce noisy or incomplete image data. Hence, before they can be evaluated by doctors, they have to be processed. Keywords in this context are image denoising, image deblurring, image decomposition and image inpainting.
One of the most successful image processing approaches are so-called partial differential equations (PDEs) and variational models. Given a noisy image, its processed (denoised) version is computed as a solution of a PDE or as a minimiser of a functional (variational model). Both of these processes are regularising the given image and herewith eliminate noise or fill missing parts in images. Favourable imaging approaches are doing so by eliminating high-frequency features (noise) while preserving or even enhancing low-frequency features (object boundaries, edges).
In this project we propose to focus on one of the most effective while least understood classes in this context: methods that involve expressions of high, especially fourth, differential order. Higher-order methods by far outperform standard image restoration algorithms in terms of the high-quality visual results they produce. Bringing together the expertises from different fields of mathematics, among them applied PDEs, variational calculus, geometric measure theory and modern numerical analysis, we attempt to answer and complement some of the many open questions evolving around higher-order imaging models.
The punchline of the project is a specific image processing task called image inpainting. Inpainting denotes the process of filling-in missing parts in an image using the information gained from the intact part of the image. It is essentially a type of interpolation and has applications, e.g., in the restoration of old photographs and paintings, text erasing (e.g., removal of dates in digital images or subtitles in a movie), or special effects like object disappearance. Adding additional geometrical constraints to this interpolation process, higher-order methods are able to address some of the shortcomings of standard inpainting methods like the ability to restore contents in very large gaps in an image.
In order to have effective and reliable higher-order inpainting approaches it is inevitable to analyse their mathematical properties thoroughly. Questions to answer are: what kind of solutions do these approaches produce? What are the characteristic features (like regularity and sparseness) they promote in the resulting image? Which terms in the mathematical setup do we have to manipulate and how, to stir the interpolation process to our liking?
Another issue is their numerical implementation. In fact, the unfortunate reason why these models are not accommodated in applied tasks and standard imaging software is that their solution with current numerical algorithms is still expensive and far away from real-time user interaction.
This project addresses the development, analysis and efficient numerical implementation of imaging models using PDEs and variational formulations of high-differential order with sophisticated tools from modern applied mathematics.
One of the most successful image processing approaches are so-called partial differential equations (PDEs) and variational models. Given a noisy image, its processed (denoised) version is computed as a solution of a PDE or as a minimiser of a functional (variational model). Both of these processes are regularising the given image and herewith eliminate noise or fill missing parts in images. Favourable imaging approaches are doing so by eliminating high-frequency features (noise) while preserving or even enhancing low-frequency features (object boundaries, edges).
In this project we propose to focus on one of the most effective while least understood classes in this context: methods that involve expressions of high, especially fourth, differential order. Higher-order methods by far outperform standard image restoration algorithms in terms of the high-quality visual results they produce. Bringing together the expertises from different fields of mathematics, among them applied PDEs, variational calculus, geometric measure theory and modern numerical analysis, we attempt to answer and complement some of the many open questions evolving around higher-order imaging models.
The punchline of the project is a specific image processing task called image inpainting. Inpainting denotes the process of filling-in missing parts in an image using the information gained from the intact part of the image. It is essentially a type of interpolation and has applications, e.g., in the restoration of old photographs and paintings, text erasing (e.g., removal of dates in digital images or subtitles in a movie), or special effects like object disappearance. Adding additional geometrical constraints to this interpolation process, higher-order methods are able to address some of the shortcomings of standard inpainting methods like the ability to restore contents in very large gaps in an image.
In order to have effective and reliable higher-order inpainting approaches it is inevitable to analyse their mathematical properties thoroughly. Questions to answer are: what kind of solutions do these approaches produce? What are the characteristic features (like regularity and sparseness) they promote in the resulting image? Which terms in the mathematical setup do we have to manipulate and how, to stir the interpolation process to our liking?
Another issue is their numerical implementation. In fact, the unfortunate reason why these models are not accommodated in applied tasks and standard imaging software is that their solution with current numerical algorithms is still expensive and far away from real-time user interaction.
This project addresses the development, analysis and efficient numerical implementation of imaging models using PDEs and variational formulations of high-differential order with sophisticated tools from modern applied mathematics.
Planned Impact
The processing of digital images experiences growing importance in many different situations. From the organisation and enhancement of digital pictures from our last holidays in Spain to the processing and professional interpretation of images coming from medical imaging tools (e.g., MRI, CT), from security cameras and satellites, image processing is ubiquitous. One major part of image processing is image restoration, where the task is to restore damaged parts in an image. In this project we shall study one of the most effective while least understood methods in this context: higher-order partial differential equations (PDEs) and variational methods. The focus of the project is to gain a better understanding of these algorithms by a thorough analysis and to efficiently implement them on the computer in order to make them accessible to the public.
The beneficiaries from this research are of course all people who deal with digital images.
In medical imaging, for instance, our methods constitute an educated post-processing of MRI images. By eliminating artefacts like noise (due to the low-resolution measurement process) they shall facilitate and accelerate the doctor's diagnose, which is based on a definite interpretation of the MRI image.
Further, algorithms for image restoration have an impact in the creative industry. Many museums nowadays digitise their art collections. Hence, it makes sense to supply them with an imaging software to manipulate paintings digitally. A digital restoration of historic paintings contributes to their conservation, their online presentation and may as well serve the conservators as a template for the physical restoration of the paintings. An example of a successful application of mathematical imaging for arts restoration is our work on the Neidhart frescoes in Vienna http://plus.maths.org/content/os/issue50/features/schoenlieb/index Moreover, high-quality imaging algorithms for a flexible manipulation of images are also interesting for visual artists, like Franz Schubert http://www.schbrt.com/
In forensics and security applications the enhancement of digital images is of outmost importance as well. Here, the restoration of low-quality fingerprints, for instance, is a very active field of research still facing many hurdles and limitations, which have to be taken.
From another perspective, image processing methods which use sophisticated mathematical concepts like PDEs establish an alternative (visual) approach towards the understanding of PDEs in general. Hence, this project also benefits the education in mathematics by making the subject more accessible. Compare also page 17 of the EPSRC Annual Report and Accounts 2009-2010.
Developed restoration algorithms in this project, have a great potential for improving the processing of images in the above mentioned applications. The current project executes some steps towards the use of these algorithms in practice. Namely, we shall work on the development and analysis of higher-order imaging techniques and on their efficient implementation on the computer. This constitutes the basis for their successful establishment in more applied fields, like medicine and forensics, and their use in standard imaging software (like Adobe Photoshop or Gimp). In the two-years research carried out in the project the applicant and the PostDoc shall have gained a deep understanding of these algorithms and their realisation to be able to directly conduct a dialogue with potential customers and software engineers.
To conclude, the project fosters the competitiveness of the UK in developing professional imaging software and in succession in improving image-guided applications in medicine, history of arts and security politics.
The beneficiaries from this research are of course all people who deal with digital images.
In medical imaging, for instance, our methods constitute an educated post-processing of MRI images. By eliminating artefacts like noise (due to the low-resolution measurement process) they shall facilitate and accelerate the doctor's diagnose, which is based on a definite interpretation of the MRI image.
Further, algorithms for image restoration have an impact in the creative industry. Many museums nowadays digitise their art collections. Hence, it makes sense to supply them with an imaging software to manipulate paintings digitally. A digital restoration of historic paintings contributes to their conservation, their online presentation and may as well serve the conservators as a template for the physical restoration of the paintings. An example of a successful application of mathematical imaging for arts restoration is our work on the Neidhart frescoes in Vienna http://plus.maths.org/content/os/issue50/features/schoenlieb/index Moreover, high-quality imaging algorithms for a flexible manipulation of images are also interesting for visual artists, like Franz Schubert http://www.schbrt.com/
In forensics and security applications the enhancement of digital images is of outmost importance as well. Here, the restoration of low-quality fingerprints, for instance, is a very active field of research still facing many hurdles and limitations, which have to be taken.
From another perspective, image processing methods which use sophisticated mathematical concepts like PDEs establish an alternative (visual) approach towards the understanding of PDEs in general. Hence, this project also benefits the education in mathematics by making the subject more accessible. Compare also page 17 of the EPSRC Annual Report and Accounts 2009-2010.
Developed restoration algorithms in this project, have a great potential for improving the processing of images in the above mentioned applications. The current project executes some steps towards the use of these algorithms in practice. Namely, we shall work on the development and analysis of higher-order imaging techniques and on their efficient implementation on the computer. This constitutes the basis for their successful establishment in more applied fields, like medicine and forensics, and their use in standard imaging software (like Adobe Photoshop or Gimp). In the two-years research carried out in the project the applicant and the PostDoc shall have gained a deep understanding of these algorithms and their realisation to be able to directly conduct a dialogue with potential customers and software engineers.
To conclude, the project fosters the competitiveness of the UK in developing professional imaging software and in succession in improving image-guided applications in medicine, history of arts and security politics.
Organisations
- University of Cambridge (Lead Research Organisation)
- Friedrich-Alexander University Erlangen-Nuremberg (Collaboration)
- University College London (Collaboration)
- Norwegian University of Science and Technology (NTNU) (Collaboration)
- La Trobe University (Collaboration)
- UNIVERSITY OF CAMBRIDGE (Collaboration)
- QUEEN MARY UNIVERSITY OF LONDON (Collaboration)
- Cancer Research UK Cambridge Institute (Collaboration)
- University of Osnabrück (Collaboration)
- École Polytechnique (Collaboration)
- University of Bath (Collaboration)
- University of Lyon (Collaboration)
- GE Healthcare Limited (Collaboration)
- Saarland University (Collaboration)
- Luebeck University of Applied Sciences (Collaboration)
- University of Münster (Collaboration)
- King Abdullah University of Science and Technology (KAUST) (Collaboration)
- CLK GmBH (Collaboration)
People |
ORCID iD |
| Carola-Bibiane Schönlieb (Principal Investigator) |
Publications
Lee J
(2015)
Nonparametric Image Registration of Airborne LiDAR, Hyperspectral and Photographic Imagery of Wooded Landscapes
in IEEE Transactions on Geoscience and Remote Sensing
Lee J
(2016)
Individual Tree Species Classification From Airborne Multisensor Imagery Using Robust PCA
in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Lellmann J
(2014)
Imaging with Kantorovich--Rubinstein Discrepancy
in SIAM Journal on Imaging Sciences
Lellmann J
(2015)
Analysis and Application of a Nonlocal Hessian
in SIAM Journal on Imaging Sciences
Lellmann J
(2013)
Total Variation Regularization for Functions with Values in a Manifold
Lellmann J
(2013)
Discrete and Continuous Models for Partitioning Problems
in International Journal of Computer Vision
Lenzen F
(2014)
Solving Quasi-Variational Inequalities for Image Restoration with Adaptive Constraint Sets
in SIAM Journal on Imaging Sciences
Lenzen F
(2013)
Scale Space and Variational Methods in Computer Vision
Li C
(2019)
Identifying Multiple Invasive Intratumor Habitats in Glioblastoma Using Multi-Parametric Magnetic Resonance Imaging and Copula Transform
in International Journal of Radiation Oncology*Biology*Physics
Li C
(2019)
P14.129 Predicting glioblastoma invasion using multiparametric MRI and a bi-level machine learning approach
in Neuro-Oncology
| Title | Making stuff disappear using applied mathematics (video) |
| Description | A fun video showing various methods for removing items from a video using applied mathematics |
| Type Of Art | Film/Video/Animation |
| Year Produced | 2019 |
| Impact | This video will reach an international audience through YouTube, engaging researchers, industry and the general public. To date over 1300 people have viewed the video. |
| URL | https://www.youtube.com/watch?v=-yfApxV62hw&feature=youtu.be |
| Title | Mathematical Analysis Can Make You Fly! |
| Description | It's not supernatural powers that help Joana fly, but the equation on the blackboard. This partial differential equation can be used to fill in specific parts of an image based on what's around it. The process is called inpainting. For this image the equation was solved numerically to remove the stool Joana was sitting on. Image inpainting has wide ranging practical applications: from the restoration of satellite images, enhancement of medical images, the renovation of digital photographs and artwork, to special effects in images and videos. This image was created by Carola-Bibiane Schönlieb, head of the Cambridge Image Analysis group at the Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Joana Grah and Kostas Papafitsoros. It won the EPSRC science photo competition in 2014. |
| Type Of Art | Image |
| Year Produced | 2014 |
| Impact | Promotion of women in mathematics Awareness of the important role of mathematical analysis in applications. |
| URL | https://plus.maths.org/content/maths-it-gives-you-wings |
| Description | In this project we develop novel higher-order regularisation techniques for image enhancement. In particular, we propose new mathematical models, analyse their qualitative and quantitative behaviour, develop numerical algorithms for their efficient computation, and probe these new techniques by applying them to real-world imaging tasks. Up to date we have the following concrete research findings: 1. We developed a very interesting third-order method for computing high-resolution digital elevation maps (DEM). Our method allows to accurately recover natural landscapes from very few line- or point measurements, with possible applications in the reconstruction of coastal landscape from satellite measurements, generation of height maps from contour data, and compression of digital elevation maps. In comparison to previous methods, sharp features such as ridges or mountain peaks are not removed. While non-convex, our numerical method seems to be very stable and reliably converges to a fixed point within few iterations. This is joint work with J. Lellmann (DAMTP, Cambridge) and J.-M. Morel (ENS, Cachan) 2. Proposal, detailed analysis and numerical discussion of a higher-order regularisation model for image denoising, deblurring and inpainting. This higher order model improves upon other models of its kind in terms of avoiding the creation of undesirable artifacts and blocky-like structures in the reconstructed images (a known disadvantage of the standard total variation model) while being simple and very efficiently numerically solvable (a disadvantage of state of the art higher-order models normally is that they consume unreasonable computational time). This is joint work with K. Papafitsoros (CCA, Cambridge), B. Sengul (CCA, Cambridge). 3. We investigate dimensional splitting schemes for nonlinear fourth-order partial differential equations employed for large-scale image denoising and image inpainting. The study of higher order PDEs is still very young and therefore both their analysis and suitable numerical solutions are challenging problems. Nonlinear fourth-order PDEs, in particular, play an important role in image processing as they constitute rich mathematical models (geometric image features encoded in derivatives and nonlinear functions) for enhancing and analysing images. One of the main obstacles of these methods, however, is their practical realisation for large-scale data which is complicated by the nonlinearity and high-differential order of these equations. We considered two prototype equations of this class which have found increasing interest in the imaging community due to their ability to process images in a taylor-made fashion, preserving and enhancing intrinsic structural features in the image while eliminating corruptions such as noise or gaps in the image. For these equations we developed two novel computational methods. The first one approximates the original equation by a sequence of lower-dimensional, simpler equations which we show to be more efficient for solving problems such as image de-noising and image in-painting (filling gaps in images) applied to images of very large size and dimension. The second method we propose is very interesting as it combines accurate mathematical modelling with real time computation by combining ideas from convex analysis with nonlinear optimisation methods. It constitutes a damped Newton method on top of a primal-dual scheme which accurately resolves the nonlinear subgradients of the non-smooth energy functional. The proposed route promises a deeper understanding of the equation dynamics which is important for tuning them towards, e.g., improved inpainting schemes. This is joint work with B. Duering (University of Sussex), M. Benning (MRRC, Cambridge) and L. Calatroni (CCA, Cambridge). 4. When assigned with the task of reconstructing an image from given data, the first challenge is the derivation of a truthful data model and a truthful image model. Such models can be determined by an a-priori knowledge about the image, the data and their relation to each other. The source of this knowledge may be either our understanding of the type of images we wish to reconstruct and of the physics behind data acquisition or an attempt to infer parametric models from the data itself. The common question arises: how can we optimise our model choice and with it maximise the quality of the reconstruction? We have developed customised image denoising methods which use mathematical optimisation techniques that can tune an image denoising method to the type of noisy image considered. This is joint work with Luca Calatroni (CCA), Juan Carlos De Los Reyes (Escuela Politecnica National de Quito) and Tuomo Valkonen (CIA, DAMTP) 5. In recent years there has been significant developments in the reconstruction of magnetic resonance velocity images from sub-sampled k-space data. While showing a strong improvement in reconstruction quality compared to classical approaches, the vast number of di erent methods, and the challenges in setting them up, often leaves the user with the difficult task of choosing the correct approach, or more importantly, not selecting a poor approach. We survey variational approaches for the reconstruction of phase-encoded magnetic resonance velocity images from sub-sampled k-space data. We are particularly interested in regularisers that correctly treat both smooth and geometric features of the image. These features are common to velocity imaging, where the flow field will be smooth but interfaces between the fluid and surrounding material will be sharp, but are challenging to represent sparsely. As an example we demonstrate the variational approaches on velocity imaging of water flowing through a packed bed of solid particles. We evaluate Wavelet regularisation against Total Variation and the relatively recent second-order Total Generalised Variation regularisation. We combine these regularisation schemes with a contrast enhancement approach and provide a good criterion for stopping the iterations. Therefore, given only the noise level, we present a robust guideline for setting up a variational reconstruction scheme for MR velocity imaging that promotes sparsity. This is joint work with Martin Benning (MRRC, Cambridge), Lynn Gladden (MRRC, Cambridge), Daniel Holland (MRRC, Cambridge), and Tuomo Valkonen (CIA, DAMTP) 6. Sparse and higher-order models in forest ecology: A major problem faced by forest conservation managers is how to efficiently survey the condition of extensive forest areas, so that informed decisions about conservation and restoration activities can be made. Remote sensing using unmanned aerial vehicles provides a promising method for the acquisition of high-quality imagery at relatively low costs, but the handling of such data presents significant technical challenges. In this project we have started to develop sparse and higher-order image processing tools for multi-modal image analysis of LIDAR, hyper spectral images and aerial photographs. This is joint work with X. Cai (Plant Sciences and CIA, Cambridge), D. Coomes (Plant Sciences, Cambridge), J. Lee (Plant Sciences, Cambridge) and J. Lellmann (CIA, Cambridge). 7. Sparse and mass preserving image restoration: In many imaging applications, foremost biomedical imaging, the image is a physical quantity, a concentration of a particular substance or biological specimen. Image enhancement techniques which are able to preserve the physical nature of the image, such as its total mass, are therefore essential for a meaningful analysis of such images. In a series of works we have developed image enhancement techniques which can do exactly this while at the same time promoting sparse geometric structures in the image. This is joint work with M. Burger (University of Münster), M. Franek, J. Lellmann (CIA, Cambridge), D. Lorenz (University of Braunschweig) and T. Valkonen (CIA, Cambridge). 8. Guided image inpainting for arts restoration: We have examined very closely state of the art inpainting techniques for their application to digital art restoration and have started to integrate our findings in an inpainting tool for museum conservators. This is joint work with M. D'Autume (ENS Cachan), S. Bucklow (Hamilton Kerr Institute, Cambridge), R. Hocking (CIA, Cambridge), S. Panayotova (Fitzwilliam, Cambridge), and P. Ricciardi (MINIARE, Cambridge). 9. Development and extensive analysis of transport-based inpainting techniques (PhD thesis of Rob Hocking which is currently under revision). 9. Anisotropic higher-order regularisation: PhD thesis of Simone Parisotto, and several papers in my personal portfolio. |
| Exploitation Route | Methods developed within this project promise to advance imaging related applications in a variety of different context such as clinical medicine, security, oil exploration, education, entertainment, and arts. In clinical medicine imaging instruments like the magnetic resonance tomograph, computer/emission tomograph, and light microscopes often constitute the main diagnostic tools for monitoring internal processes in the body. The quality of the imaging result mostly depends on the acquisition time, the exposure of the patient to radiation, as well as the capability of the image reconstruction method. Once implemented in clinical practice, our restoration algorithms have a strong potential to link into the latter point and consequently reduce the duration of the acquisition process and the amount of radiation that is used. This promises immediate improvement upon patient comfort and hospital costing. Another area of potential impact is in surveillance and forensics. Here, the enhancement of digital images is of outmost importance. The restoration of low-quality fi ngerprints, for instance, is a very active field of research still facing many hurdles and limitations, which have to be overcome. Moreover, security cameras produce huge amounts of imaging data of limited quality. The challenges in analysing and interpreting this data are in its size and in its limitations due to visualising 3D information in a two dimensional photograph or video. Similar problems as in security applications appear also in the media industry for producing special effects by making hidden objects visible or letting objects in scenes disappear. Being able to preprocess this data to improve upon these issues is a major motivation for the development of image restoration algorithms. Algorithms for image restoration also have an impact in the creative industry and the management of museum archives. Digital restoration methods are able to document losses and damages in art pieces as well as their possible repair. Insights gained from digital processing can be used for formalising restoration work but also for museum exhibitions and archive documentation. For oil exploration, seismic monitoring, and the understanding of how our climate and nature is evolving, remote sensing techniques such as RADAR, LIDAR or hyper spectral imaging are the main tools that are used. Again, being able to automatically enhance and process these images guarantees to make the most of the data being measured. From another perspective, image processing methods which use sophisticated mathematical concepts such as differential equations establish an alternative (visual) approach towards the understanding of these in general. Hence, this project also benefit s the education in mathematics by making the subject more accessible. The outcomes of the project will benefit researchers in applied mathematics, engineering, computer science and other areas of imaging and vision, e.g., art historians, medics, or biologists. The developed imaging techniques will contribute to the active and fast growing field of mathematical image processing. Our methods by far outperform standard PDE and variational methods with respect to the high-quality visual results they produce. Moreover, in terms of computational efficiency our proposed algorithms are competitive with standard techniques. The impact of these methods in the community is guaranteed through 1. The usual dissemination strategies: presentations at international conferences, publication in international journals, and a well-observed webpage with publicly available papers and algorithms. 2. Reaching out for improving Matlab imaging: In many applied sciences, such as engineering, biology, medicine, etc., the mathematical software Matlab, is used as a common programming language. Hence, for our methods to have impact within these communities it is important to publicise them on Mathworks platforms. a. Matlab Digest Article on Applying Modern PDE Techniques to Digital Image Restoration, see http://www.mathworks.co.uk/company/newsletters/articles/applying-modern-pde-techniques-to-digital-image-restoration.html b. Inpainting Codes on Matlab Central, see http://www.mathworks.co.uk/matlabcentral/fileexchange/34356 3. Collaborating with users of our methodologies: Within the project period we have initiated or intensified interdisciplinary collaborations with the Wolfson Brain Imaging Centre, the Cambridge Cancer Centre, Magnetic Resonance Research Centre and the Fitzwilliam Museum. Joint projects, initiatives and an active exchange like this is very important for both the further development of imaging methods (tuned to meet the needs for applications) and their introduction and spread in the applied sciences. |
| Sectors | Aerospace, Defence and Marine,Digital/Communication/Information Technologies (including Software),Education,Environment,Healthcare,Culture, Heritage, Museums and Collections,Security and Diplomacy,Transport |
| URL | http://www.damtp.cam.ac.uk/research/cia/ |
| Description | Our findings have been used in various interdisciplinary projects (most of which are still ongoing) with researchers in biomedical imaging, remote sensing and art conservation: 1. Art conservation: In a joint endeavour with the Fitzwilliam Museum in Cambridge we have developed digital image restoration techniques using sparse and higher-order partial differential equations. Various state of the art techniques have been tested, new restoration methods have been developed (third-order surface interpolation and GuideFill, cf. Research Findings page) and will next be implemented within a software tool for museum conservators. We are planning to launch this tool in the course of an Arts and Science exhibition in Cambridge in 2016. This is joint work with S. Bucklow (Hamilton Kerr Institute, Cambridge), M. D'Autume (ENS Cachan, Paris), R. Hocking (CIA, Cambridge), S. Panayotova (Fitzwilliam Museum, Cambridge), and P. Ricciardi (MINIARE, Cambridge). Collaboration with the Fitzwilliam Museum on virtual arts restoration which has been exhibited in the Colour exhibition in Cambridge http://www.fitzmuseum.cam.ac.uk/colour, and was featured in several public presentations I have given in the last year, the most recent ones in my Gresham lecture 2017 and in a panel discussion on `Unveiling the mysteries of science in art' at the Cambridge Science Festival 2018. We have recently received funding from the Leverhulme Trust for continuing our work on digital art restoration (funding period January 2019- December 2021). 2. High resolution imaging and analysis in biomedicine: Development and employment of image enhancement and segmentation methods for applications in MR velocity imaging and light microscopy. The former is joint work with M. Benning (MRRC, Cambridge), L. Gladden (MRRC, Cambridge), D. Holland (MRRC, Cambridge), and T. Valkonen (CIA, Cambridge). The latter is joint work with M. Burger (University of Münster), J. Grah (CIA and CRUK CI), S. Reichelt (CRUK CI) and A. Schreiner (CRUK CI). Our work in biomedicine has also contributed to the funding we received from EPSRC for the Cambridge Mathematical Imaging in Healthcare Centre. Since 2018 we have also a new collaboration with the Integrative Cancer Medicine group in Cambridge using image restoration for all-in-one cancer imaging (joint with Radiology, Cambridge). 3. Forest conservation: We have developed novel image registration and segmentation methods for the analysis of remote sensing data of large forest areas. Our methods allow the combination of three different data types for the analysis of forests at the accuracy of single trees. This is joint work with X. Cai (CIA and Plant Sciences, Cambridge), D. Coomes (Plant Sciences, Cambridge), J. Lee (Plant Sciences, Cambridge), and J. Lellmann (CIA, Cambridge). 4. |
| First Year Of Impact | 2014 |
| Sector | Agriculture, Food and Drink,Environment,Healthcare,Culture, Heritage, Museums and Collections |
| Impact Types | Cultural,Societal |
| Description | PhD student training within the Cantab Capital Institute of the Mathematics of Information |
| Geographic Reach | Multiple continents/international |
| Policy Influence Type | Influenced training of practitioners or researchers |
| URL | http://www.damtp.cam.ac.uk/user/cbs31/MoI/Welcome.html |
| Description | All in one cancer imaging optimisation using an integrated mathematical and deep learning approach |
| Amount | £821,831 (GBP) |
| Organisation | Wellcome Trust |
| Sector | Charity/Non Profit |
| Country | United Kingdom |
| Start | 01/2020 |
| End | 12/2021 |
| Description | CCI Collaborative Fund on Assessing the conservation quality of tropical forest unmanned aerial vehicles |
| Amount | £30,000 (GBP) |
| Organisation | University of Cambridge |
| Department | Cambridge Conservation Initiative |
| Sector | Academic/University |
| Country | United Kingdom |
| Start | 09/2014 |
| End | 09/2016 |
| Description | Cambridge Mathematics of Information in Healthcare (CMIH) |
| Amount | £1,295,778 (GBP) |
| Funding ID | EP/T017961/1 |
| Organisation | Engineering and Physical Sciences Research Council (EPSRC) |
| Sector | Public |
| Country | United Kingdom |
| Start | 08/2020 |
| End | 08/2023 |
| Description | Combining Knowledge And Data Driven Approaches to Inverse Imaging Problems |
| Amount | £1,240,288 (GBP) |
| Funding ID | EP/V029428/1 |
| Organisation | Engineering and Physical Sciences Research Council (EPSRC) |
| Sector | Public |
| Country | United Kingdom |
| Start | 06/2021 |
| End | 05/2026 |
| Description | Donation to build the Cantab Capital Institute for the Mathematics of Information |
| Amount | £5,000,000 (GBP) |
| Organisation | Cantab Capital Partners |
| Sector | Private |
| Country | United Kingdom |
| Start | 11/2015 |
| End | 10/2020 |
| Description | EPSRC / Isaac Newton Trust Small Grant Non-smooth geometric reconstruction for high resolution MRI imaging of uid transport in bed reactors. |
| Amount | £50,000 (GBP) |
| Organisation | University of Cambridge |
| Department | Isaac Newton Trust |
| Sector | Academic/University |
| Country | United Kingdom |
| Start | 07/2012 |
| End | 06/2013 |
| Description | EPSRC Centre for Mathematical and Statistical Analysis of Multimodal Clinical Imaging |
| Amount | £1,923,014 (GBP) |
| Funding ID | EP/N014588/1 |
| Organisation | Engineering and Physical Sciences Research Council (EPSRC) |
| Sector | Public |
| Country | United Kingdom |
| Start | 03/2016 |
| End | 02/2020 |
| Description | Isaac Newton Institute research programme on Variational methods and effective algorithms for imaging and vision |
| Amount | £80,000 (GBP) |
| Organisation | Isaac Newton Institute for Mathematical Sciences |
| Sector | Academic/University |
| Country | United Kingdom |
| Start | 08/2017 |
| End | 12/2017 |
| Description | Isaac Newton Trust Grant on Automated Contouring for Radiotherapy Treatment Planning |
| Amount | £30,000 (GBP) |
| Organisation | University of Cambridge |
| Department | Isaac Newton Trust |
| Sector | Academic/University |
| Country | United Kingdom |
| Start | 04/2015 |
| End | 12/2015 |
| Description | LMS scheme 3 award to fund meetings on Current frontiers in inverse problems from theory to applications, |
| Amount | £6,000 (GBP) |
| Organisation | London Mathematical Society |
| Sector | Academic/University |
| Country | United Kingdom |
| Start | 11/2013 |
| End | 10/2016 |
| Description | MSCA-RISE-2015 - Marie Sklodowska-Curie Research and Innovation Staff Exchange (RISE): CHiPS CHallenges in Preservation of Structure |
| Amount | € 387,000 (EUR) |
| Funding ID | 691070 |
| Organisation | European Commission H2020 |
| Sector | Public |
| Country | Belgium |
| Start | 01/2016 |
| End | 12/2019 |
| Description | MSCA-RISE-2017 - Research and Innovation Staff Exchange: NoMADS: Nonlocal Methods for Arbitrary Data Sources |
| Amount | € 1,111,500 (EUR) |
| Funding ID | 777826 |
| Organisation | European Commission H2020 |
| Sector | Public |
| Country | Belgium |
| Start | 03/2018 |
| End | 02/2022 |
| Description | Mathematical and Statistical Theory of Imaging |
| Amount | £146,400 (GBP) |
| Organisation | University of Cambridge |
| Sector | Academic/University |
| Country | United Kingdom |
| Start | 01/2017 |
| End | 12/2018 |
| Description | PET++: Improving Localisation, Diagnosis and Quantification in Clinical and Medical PET Imaging with Randomised Optimisation |
| Amount | £821,421 (GBP) |
| Funding ID | EP/S026045/1 |
| Organisation | Engineering and Physical Sciences Research Council (EPSRC) |
| Sector | Public |
| Country | United Kingdom |
| Start | 08/2019 |
| End | 08/2022 |
| Description | Philip Leverhulme Prize - Mathematical and Statistical Sciences 2017 |
| Amount | £100,000 (GBP) |
| Funding ID | https://www.leverhulme.ac.uk/philip-leverhulme-prizes/deep-inversion---towards-adaptive-physical-inversion-models-imaging |
| Organisation | The Leverhulme Trust |
| Sector | Charity/Non Profit |
| Country | United Kingdom |
| Start | 11/2018 |
| End | 10/2020 |
| Description | Postdoctoral Fellowship in The Mathematics of Information |
| Amount | £50,000 (GBP) |
| Organisation | Cognizant Technology Solutions |
| Sector | Private |
| Country | United States |
| Start | 05/2019 |
| End | 04/2020 |
| Description | Robust and Efficient Analysis Approaches of Remote Imagery for Assessing Population and Forest Health in India |
| Amount | £552,554 (GBP) |
| Funding ID | EP/T003553/1 |
| Organisation | Engineering and Physical Sciences Research Council (EPSRC) |
| Sector | Public |
| Country | United Kingdom |
| Start | 09/2019 |
| End | 09/2021 |
| Description | Royal Society International Exchange Award Nr. IE110314 High-order Compressed Sensing for Medical Imaging |
| Amount | £12,000 (GBP) |
| Funding ID | IE110314 |
| Organisation | The Royal Society |
| Sector | Charity/Non Profit |
| Country | United Kingdom |
| Start | 01/2012 |
| End | 12/2013 |
| Description | Wellcome Trust/ University of Cambridge Senior ISSF internship for the project Development of Image Analysis Algorithms for Monitoring Forest Health from Aircraft |
| Amount | £15,000 (GBP) |
| Organisation | Wellcome Trust |
| Sector | Charity/Non Profit |
| Country | United Kingdom |
| Start | 05/2014 |
| End | 03/2015 |
| Title | Research data supporting 'An Anisotropic Interaction Model for Simulating Fingerprints' |
| Description | This data contains the code and data necessary to reproduce the computational results published in 'An Anisotropic Interaction Model for Simulating Fingerprints'. |
| Type Of Material | Database/Collection of data |
| Year Produced | 2017 |
| Provided To Others? | Yes |
| Title | Research data supporting 'Pattern formation of a nonlocal, anisotropic interaction model' |
| Description | This data contains the code and data necessary to reproduce the computational results published in 'Pattern formation of a nonlocal, anisotropic interaction model'. |
| Type Of Material | Database/Collection of data |
| Year Produced | 2017 |
| Provided To Others? | Yes |
| Title | Research data supporting the publication "Choose your path wisely: gradient descent in a Bregman distance framework" |
| Description | |
| Type Of Material | Database/Collection of data |
| Year Produced | 2020 |
| Provided To Others? | Yes |
| URL | https://www.repository.cam.ac.uk/handle/1810/308472 |
| Title | Research data supporting the publication "Inverse Scale Space Decomposition". |
| Description | This dataset contains MATLAB© code for the numerical computation of the numerical examples described in Section 5.1 and Section 5.2 of the publication "Inverse Scale Space Decomposition". |
| Type Of Material | Database/Collection of data |
| Year Produced | 2017 |
| Provided To Others? | Yes |
| Title | Research data supporting the publication 'Nonlinear Spectral Image Fusion' |
| Description | This is Matlab code for the creation of image fusions based on the nonlinear spectral TV transform. The method of spectral image fusion is explained in the corresponding SSVM publication 'Nonlinear Spectral Image Fusion'. In order to run the automatic image fusion pipeline with the Obama/Reagan example as visualised in the paper, please follow the instructions in the readme.txt file in the folder 'spectralImageFusionOfFaces'. If you want to compute the spectral image fusions of Gauß and Newton in the supplementary files, please follow the instructions in the Matlab live scripts 'gaussnewton.mlx' or 'newtongauss.mlx' in the folder 'Banknote examples'. |
| Type Of Material | Database/Collection of data |
| Year Produced | 2017 |
| Provided To Others? | Yes |
| Description | Anisotropic variational models and PDEs for inverse imaging problems |
| Organisation | Luebeck University of Applied Sciences |
| Country | Germany |
| Sector | Academic/University |
| PI Contribution | In this project, we introduce a new higher-order total directional variation (TDV) regulariser for inverse imaging problems by taking into account the image gradient weighted by the structural content. Theoretical and numerical details are provided for different applications: the reconstruction of noisy images and videos, the image zooming and the interpolation of scattered surface data. The idea of using directional gradients for imaging applications is also used for the generalisation of the osmosis equation, introduced by Weickert and collaborators in 2013, to its anisotropic counter-part. Anisotropic osmosis is applied to the shadow removal problem thus improving upon the isotropic approach by avoiding the blurring artefact due to the isotropic diffusion. The main idea came from CB Schönlieb in Cambridge, and was part of Dr Simone Parisotto's PhD thesis (PhD student in Schönlieb's group). All the research work was done in collaboration with everyone involved. |
| Collaborator Contribution | Equal contribution between: Simon Masnou (Lyon), Jean-Michel Morel (Cachan), Jan Lellmann (Luebeck), Luca Calatroni (Ecole Polytechnique, Paris), and Joachim Weickert (Saarland). |
| Impact | [1] Parisotto S, Lellmann J, Masnou S, Schönlieb C-B. Higher-Order Total Directional Variation. Part I: Imaging Applications. ArXiv e-print (2018) https://arxiv.org/abs/1812.05023 [2] Parisotto S, Masnou S, Schönlieb C-B. Higher-Order Total Directional Variation. Part II: Analysis. ArXiv e-print (2018) https://arxiv.org/abs/1812.05061 [3] Parisotto S, Schönlieb C-B. Total Directional Variation for Video Denoising. ArXiv e-print (2018) https://arxiv.org/abs/1812.05063 [4] Parisotto S, Calatroni L, Caliari M, Schönlieb C-B, Weickert J. Anisotropic osmosis filtering for shadow removal in images. Inverse Problems (2019) https://doi.org/10.1088/1361-6420/ab08d2 |
| Start Year | 2014 |
| Description | Anisotropic variational models and PDEs for inverse imaging problems |
| Organisation | Saarland University |
| Country | Germany |
| Sector | Academic/University |
| PI Contribution | In this project, we introduce a new higher-order total directional variation (TDV) regulariser for inverse imaging problems by taking into account the image gradient weighted by the structural content. Theoretical and numerical details are provided for different applications: the reconstruction of noisy images and videos, the image zooming and the interpolation of scattered surface data. The idea of using directional gradients for imaging applications is also used for the generalisation of the osmosis equation, introduced by Weickert and collaborators in 2013, to its anisotropic counter-part. Anisotropic osmosis is applied to the shadow removal problem thus improving upon the isotropic approach by avoiding the blurring artefact due to the isotropic diffusion. The main idea came from CB Schönlieb in Cambridge, and was part of Dr Simone Parisotto's PhD thesis (PhD student in Schönlieb's group). All the research work was done in collaboration with everyone involved. |
| Collaborator Contribution | Equal contribution between: Simon Masnou (Lyon), Jean-Michel Morel (Cachan), Jan Lellmann (Luebeck), Luca Calatroni (Ecole Polytechnique, Paris), and Joachim Weickert (Saarland). |
| Impact | [1] Parisotto S, Lellmann J, Masnou S, Schönlieb C-B. Higher-Order Total Directional Variation. Part I: Imaging Applications. ArXiv e-print (2018) https://arxiv.org/abs/1812.05023 [2] Parisotto S, Masnou S, Schönlieb C-B. Higher-Order Total Directional Variation. Part II: Analysis. ArXiv e-print (2018) https://arxiv.org/abs/1812.05061 [3] Parisotto S, Schönlieb C-B. Total Directional Variation for Video Denoising. ArXiv e-print (2018) https://arxiv.org/abs/1812.05063 [4] Parisotto S, Calatroni L, Caliari M, Schönlieb C-B, Weickert J. Anisotropic osmosis filtering for shadow removal in images. Inverse Problems (2019) https://doi.org/10.1088/1361-6420/ab08d2 |
| Start Year | 2014 |
| Description | Anisotropic variational models and PDEs for inverse imaging problems |
| Organisation | University of Lyon |
| Country | France |
| Sector | Academic/University |
| PI Contribution | In this project, we introduce a new higher-order total directional variation (TDV) regulariser for inverse imaging problems by taking into account the image gradient weighted by the structural content. Theoretical and numerical details are provided for different applications: the reconstruction of noisy images and videos, the image zooming and the interpolation of scattered surface data. The idea of using directional gradients for imaging applications is also used for the generalisation of the osmosis equation, introduced by Weickert and collaborators in 2013, to its anisotropic counter-part. Anisotropic osmosis is applied to the shadow removal problem thus improving upon the isotropic approach by avoiding the blurring artefact due to the isotropic diffusion. The main idea came from CB Schönlieb in Cambridge, and was part of Dr Simone Parisotto's PhD thesis (PhD student in Schönlieb's group). All the research work was done in collaboration with everyone involved. |
| Collaborator Contribution | Equal contribution between: Simon Masnou (Lyon), Jean-Michel Morel (Cachan), Jan Lellmann (Luebeck), Luca Calatroni (Ecole Polytechnique, Paris), and Joachim Weickert (Saarland). |
| Impact | [1] Parisotto S, Lellmann J, Masnou S, Schönlieb C-B. Higher-Order Total Directional Variation. Part I: Imaging Applications. ArXiv e-print (2018) https://arxiv.org/abs/1812.05023 [2] Parisotto S, Masnou S, Schönlieb C-B. Higher-Order Total Directional Variation. Part II: Analysis. ArXiv e-print (2018) https://arxiv.org/abs/1812.05061 [3] Parisotto S, Schönlieb C-B. Total Directional Variation for Video Denoising. ArXiv e-print (2018) https://arxiv.org/abs/1812.05063 [4] Parisotto S, Calatroni L, Caliari M, Schönlieb C-B, Weickert J. Anisotropic osmosis filtering for shadow removal in images. Inverse Problems (2019) https://doi.org/10.1088/1361-6420/ab08d2 |
| Start Year | 2014 |
| Description | Development of Image Analysis Algorithms for Monitoring Forest Health from Aircraft |
| Organisation | University of Cambridge |
| Country | United Kingdom |
| Sector | Academic/University |
| PI Contribution | Expertise on image processing and analysis |
| Collaborator Contribution | Expertise on forest ecology and remote sensing data |
| Impact | A major problem faced by tropical forest conservation managers is how to efficiently survey the condition of extensive forest areas, so that informed decisions about conservation and restoration activities can be made. Remote sensing using unmanned aerial vehicles (UAVs) provides a promising method for the acquisition of high-quality imagery at relatively low costs, but the handling of such data presents significant technical challenges. Through the development of an image analysis tool, that can be integrated into a user-friendly open-source platform, our aim is to enable the use of UAVs for high quality, cost-efficient assessment of forest condition. Outcomes: accurate registration of aerial photographs, hyper spectral images and LIDAR data; hybrid segmentation method based on all three measurements (photograph, hyper spectral and LIDAR). Disciplines involved: Mathematics Plant Sciences |
| Start Year | 2013 |
| Description | Faster PET Reconstruction by Stochastic Optimisation |
| Organisation | Ecole Polytechnique |
| Country | France |
| Sector | Academic/University |
| PI Contribution | This project is concerned with the efficient reconstruction of positron emission tomography by means of stochastic optimisation. In the last decade, many mathematical tools have been developed that have the ability to enhance clinical imaging in various ways. On the forefront of this wave are non-smooth priors that allow the reconstruction of a smooth image but do not prohibit jumps across meaningful areas like organs in medical imaging. Beside this these new tools also allow the incorporation of a-prior structual knowledge about the solution at hand. However, most of this progress has not been translated into clinical practice as most modern algorithms are too demanding for the huge data sizes encountered. In the past, algorithms have been made "applicable" to clinical practices by only considering a subset if the data at a time. While for some models this leads to satisfactory results, in general this ad-hoc strategy may yield to spurious artefacts. Motivated by the success of similar techniques in machine learning, in this project we extend modern algorithms for imaging that can handle non-smooth priors in a rigorous way to the subset setting by means of "randomisation". While the algorithm and thus its iterates are random, the variances of these are low and converge quickly to the desired deterministic solution. The Cambridge group has contributed to the algorithm development, and the design of the PET++ project (see outputs below). |
| Collaborator Contribution | Matthias Ehrhardt at the University of Bath has lead the algorithm development for stochastic optimisation for PET, and is a co-Lead on the PET++ project. Antonin Chambolle (Ecole Polytechnique) and Peter Richtárik (KAUST) have contributed with their expertise in convex optimisation and subspace decomposition approaches. Pawel Markiewicz (UCL) has contributed with his expertise on PET and associated PET data and reconstruction codes. |
| Impact | [1] Ehrhardt, M. J., Markiewicz, P. J., Schönlieb, C.-B. (2018). Faster PET Reconstruction with Non-Smooth Priors by Randomization and Preconditioning, https://arxiv.org/abs/1808.07150 [2] Ehrhardt, M. J., Markiewicz, P. J., Richtárik, P., Schott, J., Chambolle, A. & Schönlieb, C.-B. (2017). Faster PET Reconstruction with a Stochastic Primal-Dual Hybrid Gradient Method. In Proceedings of SPIE (Vol. 10394, pp. 1-12). San Diego. http://doi.org/10.1117/12.2272946. [3] Chambolle, A., Ehrhardt, M. J., Richtárik, P., & Schönlieb, C.-B. (2017). Stochastic Primal-Dual Hybrid Gradient Algorithm with Arbitrary Sampling and Imaging Applications. to appear in SIAM Journal on Optimization. http://arxiv.org/abs/1706.04957. |
| Start Year | 2016 |
| Description | Faster PET Reconstruction by Stochastic Optimisation |
| Organisation | GE Healthcare Limited |
| Country | United Kingdom |
| Sector | Academic/University |
| PI Contribution | This project is concerned with the efficient reconstruction of positron emission tomography by means of stochastic optimisation. In the last decade, many mathematical tools have been developed that have the ability to enhance clinical imaging in various ways. On the forefront of this wave are non-smooth priors that allow the reconstruction of a smooth image but do not prohibit jumps across meaningful areas like organs in medical imaging. Beside this these new tools also allow the incorporation of a-prior structual knowledge about the solution at hand. However, most of this progress has not been translated into clinical practice as most modern algorithms are too demanding for the huge data sizes encountered. In the past, algorithms have been made "applicable" to clinical practices by only considering a subset if the data at a time. While for some models this leads to satisfactory results, in general this ad-hoc strategy may yield to spurious artefacts. Motivated by the success of similar techniques in machine learning, in this project we extend modern algorithms for imaging that can handle non-smooth priors in a rigorous way to the subset setting by means of "randomisation". While the algorithm and thus its iterates are random, the variances of these are low and converge quickly to the desired deterministic solution. The Cambridge group has contributed to the algorithm development, and the design of the PET++ project (see outputs below). |
| Collaborator Contribution | Matthias Ehrhardt at the University of Bath has lead the algorithm development for stochastic optimisation for PET, and is a co-Lead on the PET++ project. Antonin Chambolle (Ecole Polytechnique) and Peter Richtárik (KAUST) have contributed with their expertise in convex optimisation and subspace decomposition approaches. Pawel Markiewicz (UCL) has contributed with his expertise on PET and associated PET data and reconstruction codes. |
| Impact | [1] Ehrhardt, M. J., Markiewicz, P. J., Schönlieb, C.-B. (2018). Faster PET Reconstruction with Non-Smooth Priors by Randomization and Preconditioning, https://arxiv.org/abs/1808.07150 [2] Ehrhardt, M. J., Markiewicz, P. J., Richtárik, P., Schott, J., Chambolle, A. & Schönlieb, C.-B. (2017). Faster PET Reconstruction with a Stochastic Primal-Dual Hybrid Gradient Method. In Proceedings of SPIE (Vol. 10394, pp. 1-12). San Diego. http://doi.org/10.1117/12.2272946. [3] Chambolle, A., Ehrhardt, M. J., Richtárik, P., & Schönlieb, C.-B. (2017). Stochastic Primal-Dual Hybrid Gradient Algorithm with Arbitrary Sampling and Imaging Applications. to appear in SIAM Journal on Optimization. http://arxiv.org/abs/1706.04957. |
| Start Year | 2016 |
| Description | Faster PET Reconstruction by Stochastic Optimisation |
| Organisation | King Abdullah University of Science and Technology (KAUST) |
| Country | Saudi Arabia |
| Sector | Academic/University |
| PI Contribution | This project is concerned with the efficient reconstruction of positron emission tomography by means of stochastic optimisation. In the last decade, many mathematical tools have been developed that have the ability to enhance clinical imaging in various ways. On the forefront of this wave are non-smooth priors that allow the reconstruction of a smooth image but do not prohibit jumps across meaningful areas like organs in medical imaging. Beside this these new tools also allow the incorporation of a-prior structual knowledge about the solution at hand. However, most of this progress has not been translated into clinical practice as most modern algorithms are too demanding for the huge data sizes encountered. In the past, algorithms have been made "applicable" to clinical practices by only considering a subset if the data at a time. While for some models this leads to satisfactory results, in general this ad-hoc strategy may yield to spurious artefacts. Motivated by the success of similar techniques in machine learning, in this project we extend modern algorithms for imaging that can handle non-smooth priors in a rigorous way to the subset setting by means of "randomisation". While the algorithm and thus its iterates are random, the variances of these are low and converge quickly to the desired deterministic solution. The Cambridge group has contributed to the algorithm development, and the design of the PET++ project (see outputs below). |
| Collaborator Contribution | Matthias Ehrhardt at the University of Bath has lead the algorithm development for stochastic optimisation for PET, and is a co-Lead on the PET++ project. Antonin Chambolle (Ecole Polytechnique) and Peter Richtárik (KAUST) have contributed with their expertise in convex optimisation and subspace decomposition approaches. Pawel Markiewicz (UCL) has contributed with his expertise on PET and associated PET data and reconstruction codes. |
| Impact | [1] Ehrhardt, M. J., Markiewicz, P. J., Schönlieb, C.-B. (2018). Faster PET Reconstruction with Non-Smooth Priors by Randomization and Preconditioning, https://arxiv.org/abs/1808.07150 [2] Ehrhardt, M. J., Markiewicz, P. J., Richtárik, P., Schott, J., Chambolle, A. & Schönlieb, C.-B. (2017). Faster PET Reconstruction with a Stochastic Primal-Dual Hybrid Gradient Method. In Proceedings of SPIE (Vol. 10394, pp. 1-12). San Diego. http://doi.org/10.1117/12.2272946. [3] Chambolle, A., Ehrhardt, M. J., Richtárik, P., & Schönlieb, C.-B. (2017). Stochastic Primal-Dual Hybrid Gradient Algorithm with Arbitrary Sampling and Imaging Applications. to appear in SIAM Journal on Optimization. http://arxiv.org/abs/1706.04957. |
| Start Year | 2016 |
| Description | Faster PET Reconstruction by Stochastic Optimisation |
| Organisation | University College London |
| Country | United Kingdom |
| Sector | Academic/University |
| PI Contribution | This project is concerned with the efficient reconstruction of positron emission tomography by means of stochastic optimisation. In the last decade, many mathematical tools have been developed that have the ability to enhance clinical imaging in various ways. On the forefront of this wave are non-smooth priors that allow the reconstruction of a smooth image but do not prohibit jumps across meaningful areas like organs in medical imaging. Beside this these new tools also allow the incorporation of a-prior structual knowledge about the solution at hand. However, most of this progress has not been translated into clinical practice as most modern algorithms are too demanding for the huge data sizes encountered. In the past, algorithms have been made "applicable" to clinical practices by only considering a subset if the data at a time. While for some models this leads to satisfactory results, in general this ad-hoc strategy may yield to spurious artefacts. Motivated by the success of similar techniques in machine learning, in this project we extend modern algorithms for imaging that can handle non-smooth priors in a rigorous way to the subset setting by means of "randomisation". While the algorithm and thus its iterates are random, the variances of these are low and converge quickly to the desired deterministic solution. The Cambridge group has contributed to the algorithm development, and the design of the PET++ project (see outputs below). |
| Collaborator Contribution | Matthias Ehrhardt at the University of Bath has lead the algorithm development for stochastic optimisation for PET, and is a co-Lead on the PET++ project. Antonin Chambolle (Ecole Polytechnique) and Peter Richtárik (KAUST) have contributed with their expertise in convex optimisation and subspace decomposition approaches. Pawel Markiewicz (UCL) has contributed with his expertise on PET and associated PET data and reconstruction codes. |
| Impact | [1] Ehrhardt, M. J., Markiewicz, P. J., Schönlieb, C.-B. (2018). Faster PET Reconstruction with Non-Smooth Priors by Randomization and Preconditioning, https://arxiv.org/abs/1808.07150 [2] Ehrhardt, M. J., Markiewicz, P. J., Richtárik, P., Schott, J., Chambolle, A. & Schönlieb, C.-B. (2017). Faster PET Reconstruction with a Stochastic Primal-Dual Hybrid Gradient Method. In Proceedings of SPIE (Vol. 10394, pp. 1-12). San Diego. http://doi.org/10.1117/12.2272946. [3] Chambolle, A., Ehrhardt, M. J., Richtárik, P., & Schönlieb, C.-B. (2017). Stochastic Primal-Dual Hybrid Gradient Algorithm with Arbitrary Sampling and Imaging Applications. to appear in SIAM Journal on Optimization. http://arxiv.org/abs/1706.04957. |
| Start Year | 2016 |
| Description | Faster PET Reconstruction by Stochastic Optimisation |
| Organisation | University of Bath |
| Country | United Kingdom |
| Sector | Academic/University |
| PI Contribution | This project is concerned with the efficient reconstruction of positron emission tomography by means of stochastic optimisation. In the last decade, many mathematical tools have been developed that have the ability to enhance clinical imaging in various ways. On the forefront of this wave are non-smooth priors that allow the reconstruction of a smooth image but do not prohibit jumps across meaningful areas like organs in medical imaging. Beside this these new tools also allow the incorporation of a-prior structual knowledge about the solution at hand. However, most of this progress has not been translated into clinical practice as most modern algorithms are too demanding for the huge data sizes encountered. In the past, algorithms have been made "applicable" to clinical practices by only considering a subset if the data at a time. While for some models this leads to satisfactory results, in general this ad-hoc strategy may yield to spurious artefacts. Motivated by the success of similar techniques in machine learning, in this project we extend modern algorithms for imaging that can handle non-smooth priors in a rigorous way to the subset setting by means of "randomisation". While the algorithm and thus its iterates are random, the variances of these are low and converge quickly to the desired deterministic solution. The Cambridge group has contributed to the algorithm development, and the design of the PET++ project (see outputs below). |
| Collaborator Contribution | Matthias Ehrhardt at the University of Bath has lead the algorithm development for stochastic optimisation for PET, and is a co-Lead on the PET++ project. Antonin Chambolle (Ecole Polytechnique) and Peter Richtárik (KAUST) have contributed with their expertise in convex optimisation and subspace decomposition approaches. Pawel Markiewicz (UCL) has contributed with his expertise on PET and associated PET data and reconstruction codes. |
| Impact | [1] Ehrhardt, M. J., Markiewicz, P. J., Schönlieb, C.-B. (2018). Faster PET Reconstruction with Non-Smooth Priors by Randomization and Preconditioning, https://arxiv.org/abs/1808.07150 [2] Ehrhardt, M. J., Markiewicz, P. J., Richtárik, P., Schott, J., Chambolle, A. & Schönlieb, C.-B. (2017). Faster PET Reconstruction with a Stochastic Primal-Dual Hybrid Gradient Method. In Proceedings of SPIE (Vol. 10394, pp. 1-12). San Diego. http://doi.org/10.1117/12.2272946. [3] Chambolle, A., Ehrhardt, M. J., Richtárik, P., & Schönlieb, C.-B. (2017). Stochastic Primal-Dual Hybrid Gradient Algorithm with Arbitrary Sampling and Imaging Applications. to appear in SIAM Journal on Optimization. http://arxiv.org/abs/1706.04957. |
| Start Year | 2016 |
| Description | Flow of Microtubules in the Drosophila Oocyte |
| Organisation | CLK GmBH |
| Country | Germany |
| Sector | Private |
| PI Contribution | The focus of this project is to characterise directionality of plus ends of microtubules in confocal microscopy images of Drosophilia embryos. This goal is particularly challenging due to the high noise level in such data, making it almost impossible to distinguish EB1 fluorescently labelled comets from randomly distributed noise. To overcome this problem, we employ recently developed methods for joint motion estimation and image reconstruction. As a result we are able to estimate motion in image sequences where other state of the art methods fail. My group in Cambridge (in particular my PostDoc Dr Lukas Lang) has contributed the algorithm development for the motion analysis (optical flow). |
| Collaborator Contribution | Our collaborators Isabel Palacios (Queen Mary London) and Mail Drechsler (University of Osnabrück) have contributed with the biological question and the imaging data. Our collaborators Martin Burger (University of Erlangen) and Hendrik Dirks (CLK GmbH) have contributed to initial ideas for the motion estimation algorithm. |
| Impact | M. Drechsler, L. F. Lang, H. Dirks, M. Burger, C.-B. Schönlieb, I. M. Palacios. Optical flow analysis reveals that Kinesin-mediated advection impacts on the orientation of microtubules, submitted, 2019. Preprint: https://www.biorxiv.org/content/10.1101/556043v2 Code: https://zenodo.org/record/2573254#.XIduWS2cZ0s |
| Start Year | 2015 |
| Description | Flow of Microtubules in the Drosophila Oocyte |
| Organisation | Friedrich-Alexander University Erlangen-Nuremberg |
| Country | Germany |
| Sector | Academic/University |
| PI Contribution | The focus of this project is to characterise directionality of plus ends of microtubules in confocal microscopy images of Drosophilia embryos. This goal is particularly challenging due to the high noise level in such data, making it almost impossible to distinguish EB1 fluorescently labelled comets from randomly distributed noise. To overcome this problem, we employ recently developed methods for joint motion estimation and image reconstruction. As a result we are able to estimate motion in image sequences where other state of the art methods fail. My group in Cambridge (in particular my PostDoc Dr Lukas Lang) has contributed the algorithm development for the motion analysis (optical flow). |
| Collaborator Contribution | Our collaborators Isabel Palacios (Queen Mary London) and Mail Drechsler (University of Osnabrück) have contributed with the biological question and the imaging data. Our collaborators Martin Burger (University of Erlangen) and Hendrik Dirks (CLK GmbH) have contributed to initial ideas for the motion estimation algorithm. |
| Impact | M. Drechsler, L. F. Lang, H. Dirks, M. Burger, C.-B. Schönlieb, I. M. Palacios. Optical flow analysis reveals that Kinesin-mediated advection impacts on the orientation of microtubules, submitted, 2019. Preprint: https://www.biorxiv.org/content/10.1101/556043v2 Code: https://zenodo.org/record/2573254#.XIduWS2cZ0s |
| Start Year | 2015 |
| Description | Flow of Microtubules in the Drosophila Oocyte |
| Organisation | Queen Mary University of London |
| Country | United Kingdom |
| Sector | Academic/University |
| PI Contribution | The focus of this project is to characterise directionality of plus ends of microtubules in confocal microscopy images of Drosophilia embryos. This goal is particularly challenging due to the high noise level in such data, making it almost impossible to distinguish EB1 fluorescently labelled comets from randomly distributed noise. To overcome this problem, we employ recently developed methods for joint motion estimation and image reconstruction. As a result we are able to estimate motion in image sequences where other state of the art methods fail. My group in Cambridge (in particular my PostDoc Dr Lukas Lang) has contributed the algorithm development for the motion analysis (optical flow). |
| Collaborator Contribution | Our collaborators Isabel Palacios (Queen Mary London) and Mail Drechsler (University of Osnabrück) have contributed with the biological question and the imaging data. Our collaborators Martin Burger (University of Erlangen) and Hendrik Dirks (CLK GmbH) have contributed to initial ideas for the motion estimation algorithm. |
| Impact | M. Drechsler, L. F. Lang, H. Dirks, M. Burger, C.-B. Schönlieb, I. M. Palacios. Optical flow analysis reveals that Kinesin-mediated advection impacts on the orientation of microtubules, submitted, 2019. Preprint: https://www.biorxiv.org/content/10.1101/556043v2 Code: https://zenodo.org/record/2573254#.XIduWS2cZ0s |
| Start Year | 2015 |
| Description | Flow of Microtubules in the Drosophila Oocyte |
| Organisation | University of Osnabrück |
| Country | Germany |
| Sector | Academic/University |
| PI Contribution | The focus of this project is to characterise directionality of plus ends of microtubules in confocal microscopy images of Drosophilia embryos. This goal is particularly challenging due to the high noise level in such data, making it almost impossible to distinguish EB1 fluorescently labelled comets from randomly distributed noise. To overcome this problem, we employ recently developed methods for joint motion estimation and image reconstruction. As a result we are able to estimate motion in image sequences where other state of the art methods fail. My group in Cambridge (in particular my PostDoc Dr Lukas Lang) has contributed the algorithm development for the motion analysis (optical flow). |
| Collaborator Contribution | Our collaborators Isabel Palacios (Queen Mary London) and Mail Drechsler (University of Osnabrück) have contributed with the biological question and the imaging data. Our collaborators Martin Burger (University of Erlangen) and Hendrik Dirks (CLK GmbH) have contributed to initial ideas for the motion estimation algorithm. |
| Impact | M. Drechsler, L. F. Lang, H. Dirks, M. Burger, C.-B. Schönlieb, I. M. Palacios. Optical flow analysis reveals that Kinesin-mediated advection impacts on the orientation of microtubules, submitted, 2019. Preprint: https://www.biorxiv.org/content/10.1101/556043v2 Code: https://zenodo.org/record/2573254#.XIduWS2cZ0s |
| Start Year | 2015 |
| Description | Geometric Integration Methods for Optimisation |
| Organisation | La Trobe University |
| Country | Australia |
| Sector | Academic/University |
| PI Contribution | This project is concerned with the development and analysis of optimisation schemes based on geometric numerical integration methods. Discrete gradient methods are popular numerical schemes for solving systems of ODEs, and are known for preserving structures of the continuous system such as energy dissipation/conservation. Applying discrete gradients to dissipative ODEs/PDEs yields optimisation schemes that preserve the dissipative structure. For example, we consider a derivative-free discrete gradient method for optimising nonsmooth, nonconvex problems in a blackbox setting. This method has been shown to converge to optimal points of the objective function in a general, nonsmooth setting, while retaining favourable properties of gradient flow. This blackbox optimisation framework is useful, for instance, for bilevel optimisation of regularisation parameters in image processing. We (my PostDoc Matthias Ehrhardt and myself) have developed the idea and this topic in Cambridge, and my PhD student Erlend Riis has done all the analysis and numerical tests that are contained in thee papers (see below). |
| Collaborator Contribution | Our collaborator Reinout Quispel (La Trobe, Melbourne, Australia) has contributed expertise in geometric integration. Our collaborator Torbjørn Ringholm (NTNU, Trondheim, Norway) has contributed to the convergence analysis in the smooth case, and was the lead author on the Euler elastica paper. Our collaborator Jasmina Lazic (former MathWorks Cambridge) has contributed to the parallelisation of the discrete gradient method. |
| Impact | E. S. Riis, M. J. Ehrhardt, G. R. W. Quispel, and C.-B. Schönlieb, A geometric integration approach to nonsmooth, nonconvex optimisation, arXiv:1807.07554, 2018. M. Ehrhardt, E. Riis, T. Ringholm, and C.-B. Schönlieb, A geometric integration approach to smooth optimisation: Foundations of the discrete gradient method, arXiv:1805.06444, 2018. T. Ringholm, J. Lazic, C.-B. Schönlieb, Variational image regularization with Euler's elastica using a discrete gradient scheme, SIAM J. Imaging Sci., 11(4), 2665-2691, 2018. Exhibition at MATLAB Expo in Silverstone in 2017. |
| Start Year | 2017 |
| Description | Geometric Integration Methods for Optimisation |
| Organisation | Norwegian University of Science and Technology (NTNU) |
| Country | Norway |
| Sector | Academic/University |
| PI Contribution | This project is concerned with the development and analysis of optimisation schemes based on geometric numerical integration methods. Discrete gradient methods are popular numerical schemes for solving systems of ODEs, and are known for preserving structures of the continuous system such as energy dissipation/conservation. Applying discrete gradients to dissipative ODEs/PDEs yields optimisation schemes that preserve the dissipative structure. For example, we consider a derivative-free discrete gradient method for optimising nonsmooth, nonconvex problems in a blackbox setting. This method has been shown to converge to optimal points of the objective function in a general, nonsmooth setting, while retaining favourable properties of gradient flow. This blackbox optimisation framework is useful, for instance, for bilevel optimisation of regularisation parameters in image processing. We (my PostDoc Matthias Ehrhardt and myself) have developed the idea and this topic in Cambridge, and my PhD student Erlend Riis has done all the analysis and numerical tests that are contained in thee papers (see below). |
| Collaborator Contribution | Our collaborator Reinout Quispel (La Trobe, Melbourne, Australia) has contributed expertise in geometric integration. Our collaborator Torbjørn Ringholm (NTNU, Trondheim, Norway) has contributed to the convergence analysis in the smooth case, and was the lead author on the Euler elastica paper. Our collaborator Jasmina Lazic (former MathWorks Cambridge) has contributed to the parallelisation of the discrete gradient method. |
| Impact | E. S. Riis, M. J. Ehrhardt, G. R. W. Quispel, and C.-B. Schönlieb, A geometric integration approach to nonsmooth, nonconvex optimisation, arXiv:1807.07554, 2018. M. Ehrhardt, E. Riis, T. Ringholm, and C.-B. Schönlieb, A geometric integration approach to smooth optimisation: Foundations of the discrete gradient method, arXiv:1805.06444, 2018. T. Ringholm, J. Lazic, C.-B. Schönlieb, Variational image regularization with Euler's elastica using a discrete gradient scheme, SIAM J. Imaging Sci., 11(4), 2665-2691, 2018. Exhibition at MATLAB Expo in Silverstone in 2017. |
| Start Year | 2017 |
| Description | Methods for automatic mitosis detection and tracking in phase contrast images |
| Organisation | Cancer Research UK Cambridge Institute |
| Country | United Kingdom |
| Sector | Academic/University |
| PI Contribution | Detection and segmentation/tracking methodology |
| Collaborator Contribution | Master student exchange with the University of Münster; tracking methodology Cancer Research UK, CI developed the research question and has contributed the data and expertise to interpret the analysis results. |
| Impact | Nowadays, biomedical sciences and more specifically mitotic index analysis in cancer research strongly depend on evaluation and processing of digital microscopy images. This thesis deals with mathematical analysis of phase contrast microscopy images and presents methods for automatic detection and tracking of mitotic cells. We briefly explain the biological background and provide an introduction to mathematical methods of image processing. The main part comprises illustration of established tracking methods as well as a new approach to combine methods for our specific setting within the framework of a variational problem. After discussing some results, we give a summary and an outlook. Finally, we present a Graphical User Interface which facilitates applying the presented methods in MATLAB. Outcomes: master thesis, see http://wwwmath.uni-muenster.de/num/Arbeitsgruppen/ag_burger/teaching/Master/MA%20Grah.pdf and MATLAB GUI for mitosis detection and analysis. Publication is in preparation. |
| Start Year | 2013 |
| Description | Methods for automatic mitosis detection and tracking in phase contrast images |
| Organisation | University of Münster |
| Country | Germany |
| Sector | Academic/University |
| PI Contribution | Detection and segmentation/tracking methodology |
| Collaborator Contribution | Master student exchange with the University of Münster; tracking methodology Cancer Research UK, CI developed the research question and has contributed the data and expertise to interpret the analysis results. |
| Impact | Nowadays, biomedical sciences and more specifically mitotic index analysis in cancer research strongly depend on evaluation and processing of digital microscopy images. This thesis deals with mathematical analysis of phase contrast microscopy images and presents methods for automatic detection and tracking of mitotic cells. We briefly explain the biological background and provide an introduction to mathematical methods of image processing. The main part comprises illustration of established tracking methods as well as a new approach to combine methods for our specific setting within the framework of a variational problem. After discussing some results, we give a summary and an outlook. Finally, we present a Graphical User Interface which facilitates applying the presented methods in MATLAB. Outcomes: master thesis, see http://wwwmath.uni-muenster.de/num/Arbeitsgruppen/ag_burger/teaching/Master/MA%20Grah.pdf and MATLAB GUI for mitosis detection and analysis. Publication is in preparation. |
| Start Year | 2013 |
| Description | Non-smooth geometric reconstruction for high resolution MRI imaging of fluid transport in bed reactors |
| Organisation | University of Cambridge |
| Department | Magnetic Resonance Research Centre |
| Country | United Kingdom |
| Sector | Academic/University |
| PI Contribution | Mathematical methods for variational regularisation applied to image retrieval from velocity-encoded MR measurements. |
| Collaborator Contribution | Application expertise, MRI data, funding of one PostDoc for one year |
| Impact | http://www.ceb.cam.ac.uk/directory/martin-benning |
| Start Year | 2012 |
| Description | Unveiling the invisible: mathematical methods for cultural heritage |
| Organisation | University of Cambridge |
| Department | The Fitzwilliam Museum |
| Country | United Kingdom |
| Sector | Academic/University |
| PI Contribution | Hypotheses to be tested: In-depth mathematical analysis of imaging data, developed through our collaboration, could transform the ways in which hypotheses in the study of material culture are tested by searching through algorithmically examined data collected by researchers, revealing hidden patterns in paintings, manuscripts and archaeological objects. Project objectives: We will bring cutting-edge mathematical research to the arts and humanities by focusing on three challenging problems: Textural analysis of cross-sections of paint; Virtual restoration of illuminated manuscripts; Classification of Roman pottery. We will also develop an intuitive software package that will make our methodology accessible to a wide range of arts and humanities scholars. My Cambridge group - in particular my PostDocs Dr Kasia Torgonska and Dr Simone Parisotto are developing the mathematical algorithms for all three cultural heritage applications. They will also be the main developers of the modular toolkit. |
| Collaborator Contribution | Dr Launaro (Faculty of Classics, Cambridge) collaborates on the pottery classification part of the project. With his extensive experience in Roman pottery, their excavation, their historical context, their classification and curation, he is key in defining the shape characteristics in pottery (crucial for their automated classification), in formulating the relevant questions in the different stages of the work, and in critically evaluating the results. He is also responsible for steering the shape analysis and classification capabilities of the modular toolkit and making it user-friendly for a wider use by pottery specialists. Dr Spike Bucklow (Hamilton Kerr Institute (HKI), Cambridge) collaborates on the paint cross section classification, the manuscript restoration and provides input to the modular toolkit, helping define user requirements. He provides the conservation and restoration expertise, in particular on the significance of historic pigments, paints and artists' methods etc. for the manuscript restoration and the art historic significance of technical imagery for the paint cross sections. He has curated the existing database of paint cross-section images at HKI, oversees its development in support the automated cross section classification approaches and supervises its use as a test-bed for the modular toolkit. He is driving all the research questions asked of automated analysis in the manuscript restoration, identifying key visual features of interest in digital images of cross-sections and assessing the relevance and efficiency of software-based recognition of those visual features. As specific discriminatory criteria evolve, he advises on their employment, prioritizing sequences of queries and assessing navigation of the data set. He also advises on how the developed clustering techniques for the paint cross section should be integrated in the modular toolkit. Dr Panayotova (Fitzwilliam Museum) provides the expertise on the illuminated manuscripts through her research on over 4000 illuminated manuscripts at the Fitzwilliam Museum and the Cambridge Colleges. At the start of the project, she identified damaged images and prioritised the most important examples in dated and localised manuscripts for mathematical reconstructions. She selects and provides access to c. 200 from the over 35,000 digital images available from manuscripts in the Fitzwilliam Museum and the Colleges, and c. 100 images acquired with multispectral imaging techniques by the Museum's Research Scientist (at no cost to this project). In the course of the project, she also collaborates on the cross section classification, advising on the historical and artistic background of the selected manuscripts, and providing information on the circumstances of the images' original production and subsequent damage. This represents the human-expert knowledge that will be integrated in the automated restoration. She is giving feedback on the art restoration results, advising on the optimal extent of restoration required to maximise the research potential of the original images. Moreover, Dr Panayotova is co-developing the modular-toolkit, in particular influencing its functionality and design. |
| Impact | Calatroni, Luca; Marie d'Autume and, Rob Hocking ; Panayotova, Stella; Parisotto, Simone; Ricciardi, Paola; Schönlieb, Carola-Bibiane Unveiling the invisible: mathematical methods for restoring and interpreting illuminated manuscripts Journal Article In: Heritage Science, 6 (1), pp. 56, 2018. Parisotto, Simone; Calatroni, Luca; Daffara, Claudia Digital Cultural Heritage imaging via osmosis filtering Inproceedings In: Mansouri A. El Moataz A., Nouboud Mammass F D (Ed.): ICISP 2018: Image and Signal Processing, pp. Springer, 2018. Daffara, Claudia; Parisotto, Simone; Ambrosini, Dario A multipurpose, dual-mode imaging in the MWIR range for artwork diagnostic: a systematic approach Journal Article In: Optics and Lasers in Engineering, 2017. Daffara, Claudia; Parisotto, Simone; Mariotti, Paola Ilaria Mid-infrared thermal imaging for an effective mapping of surface materials and sub-surface detachments in mural paintings: integration of thermography and thermal quasi-reflectography Inproceedings In: Optics for Arts, Architecture, and Archaeology, International Society for Optics and Photonics 2015. Leverhulme Trust project on Unveiling the Invisible - 3years from January 2019; GBP 250K. |
| Start Year | 2013 |
| Title | Anisotropic osmosis filtering for shadow removal in images |
| Description | We present an anisotropic extension of the isotropic osmosis model that has been introduced by Weickert et al. for visual computing applications, and we adapt it specifically to shadow removal applications. We show that in the integrable setting, linear anisotropic osmosis minimises an energy that involves a suitable quadratic form which models local directional structures. In our shadow removal applications we estimate the local structure via a modified tensor voting approach and use this information within an anisotropic diffusion inpainting that resembles edge-enhancing anisotropic diffusion inpainting. Our numerical scheme combines the nonnegativity preserving stencil of Fehrenbach and Mirebeau with an exact time stepping based on highly accurate polynomial approximations of the matrix exponential. The resulting anisotropic model is tested on several synthetic and natural images corrupted by constant shadows. We show that it outperforms isotropic osmosis, since it does not suffer from blurring artefacts at the shadow boundaries. |
| Type Of Technology | Software |
| Year Produced | 2018 |
| Open Source License? | Yes |
| Impact | Significantly improved shadow removal in digital images; reduces artefacts. |
| URL | https://iopscience.iop.org/article/10.1088/1361-6420/ab08d2 |
| Title | Learning Filter Functions in Regularisers by Minimising Quotients |
| Description | Learning approaches have recently become very popular in the field of inverse problems. A large variety of methods has been established in recent years, ranging from bi-level learning to high-dimensional machine learning techniques. Most learning approaches, however, only aim at fitting parametrised models to favourable training data whilst ignoring misfit training data completely. In this paper, we follow up on the idea of learning parametrised regularisation functions by quotient minimisation as established in [2]. We extend the model therein to include higher-dimensionalearning filtersl filter functions to be learned and allow for fit- and misfit-training data consisting of multiple functions. We first present results resembling behaviour of well-established derivative-based sparse regularisers like total variation or higher-order total variation in one-dimension. Our second and main contribution is the introduction of novel families of non-derivative-based regularisers. This is accomplished by learning favourable scales and geometric properties while at the same time avoiding unfavourable ones. Reference: [1] Martin Benning, Guy Gilboa, Joana Sarah Grah and Carola-Bibiane Schönlieb. "Learning Filter Functions in Regularisers by Minimising Quotients." Scale Space and Variational Methods in Computer Vision (2017), accepted. [2] Martin Benning, Guy Gilboa, and Carola-Bibiane Schönlieb. "Learning parametrised regularisation functions via quotient minimisation." PAMM 16.1 (2016): 933-936. |
| Type Of Technology | Software |
| Year Produced | 2017 |
| Open Source License? | Yes |
| Impact | Translation of research on learned sparse regularisers to the public |
| URL | https://www.repository.cam.ac.uk/handle/1810/263468 |
| Description | ATI scoping workshop on Data-Rich Phenomena - Modelling, Analysing and Simulations using Partial Differential Equations |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Other audiences |
| Results and Impact | Background This event was one of a number of scientific scoping workshops funded by the Alan Turing Institute. These workshops have helped to define the research programme at the Alan Turing Institute, whose mission is to undertake data science research at the intersection of computer science, mathematics, statistics and systems engineering. It aims to provide technically informed advice to policy makers and enable researchers from industry and academia to work together towards practical applications and solutions. Partial differential equations (PDE) are equations involving functions and their derivatives - they have applications in the natural and social sciences, engineering, computer science and economics. Today they have found their way into the data sciences, in a variety of ways. PDE models are used directly, for example for data assimilation, image processing, image analysis, shape analysis, inverse problems, computer vision, and modelling complex phenomena such as crowd motion, opinion formation, and option pricing. PDE ideas also serve as an inspiration to formulate and solve data problems on graphs and networks, such as in the use of graph-discretised PDE for various classification and community detection problems, and in the application of Gamma-convergence to link graph and continuum variational classification models (which is crucial for analysing these models and for understanding their scalability). Aims and Objectives This workshop brought together expert mathematicians and statisticians, working on nonlinear, nonlocal, and stochastic PDE models and on large, complex network problems, with industrial and academic data science users. By encouraging discussion among the participants in informal presentations and breakout sessions, it helped identify the most promising research directions combining PDE and data science. Some of the key scientific questions addressed were: Identification of the most promising directions of novel PDE approaches in data science Which data science problems are appropriate for PDE techniques Which numerical PDE based approaches can lead to scalable algorithms that are applicable to very large data sets The use of model based PDE or variational inverse problems for dimension reduction (simplification!) of high dimensional data sets Pinpointing simple, analysable PDE models capable of describing complex data phenomena (e.g. pattern formation, aggregation, transport, drift, diffusion) Using stochastic PDEs to assimilate new data into existing nonlinear or nonlocal models The workshop brought together industrial and academic experts from a diverse set of backgrounds. It explored the potential of PDEs in data science areas, such as data assimilation, data analytics and topological data analysis. This event will be of particular interest to those from the following identified application areas. Healthcare industries including medical, biotechnology Retail/consumer and business analytics Energy and manufacturing Government/public sector Transport Financial Gaming industry |
| Year(s) Of Engagement Activity | 2015 |
| URL | http://www.turing-gateway.cam.ac.uk/drp_dec2015.shtml |
| Description | Celebrating Chris Budd: A leader in mathematical innovation turns 60 |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | On the 4th of March 2020 we came together at the University of Bath and celebrate the 60th birthday of our colleague, friend and collaborator Chris Budd. The aim of this workshop was to bring together collaborators, former PhD students and PostDocs of Chris to celebrate him and his achievements in mathematical innovation over his long and fruitful academic career. |
| Year(s) Of Engagement Activity | 2019 |
| URL | https://sites.google.com/view/chrisbudd60/home |
| Description | General meeting of the European Women in Mathematics Association, 3-7 September 2018, Graz, Austria. Co-organisers: K. Baur, K. Hess, E. Resmerita and S. Terracini. |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Postgraduate students |
| Results and Impact | General Meeting of the European Women in Mathematics Association |
| Year(s) Of Engagement Activity | 2018 |
| URL | https://sites.google.com/site/ewmgm18/ |
| Description | Horizon Spotlight on Imaging |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Industry/Business |
| Results and Impact | From visualising microscopic cells to massive galaxies, imaging is a core tool for many research fields today, and it's also the basis of a surge in recent technical developments - some of which are being pioneered in Cambridge. Much of the excitement around imaging is linked to its remarkably cross-disciplinary nature. IMAGES, a group of developers and users of imaging and analysis tools, is helping these researchers to work together. Dr Carola-Bibiane Schönlieb, Department of Applied Mathematics and Theoretical Physics; Dr Stella Panayotova, Fitzwilliam Museum Cambridge; Dr Stefanie Reichelt, Cancer Research UK Cambridge Institute |
| Year(s) Of Engagement Activity | 2015 |
| URL | http://www.cam.ac.uk/research/spotlight-on/imaging |
| Description | ICMS Workshop on Gradient flows: from theory to application |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Other audiences |
| Results and Impact | The main objective of this workshop is to bring experts from analysis, numerics and relevant applications together, initiate intra- and interdisciplinary collaborations and develop pathways from analytical and numerical methods into real-world applications. The workshop will feature bridging talks and several discussion rounds over the week. The workshop aims to advance and improve the knowledge on gradient flows in established and novel research directions. |
| Year(s) Of Engagement Activity | 2015 |
| URL | http://www.icms.org.uk/workshop.php?id=336 |
| Description | IMA Conference on Inverse Problems: From Theory To Application |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | This conference brought together mathematicians and statisticians, working on theoretical and numerical aspects of inverse problems, as well as engineers, physicists, and other scientists, working on challenging inverse problem applications. We welcomed industrial representatives, doctoral students, early career and established academics working in this field to attend. |
| Year(s) Of Engagement Activity | 2019 |
| URL | https://ima.org.uk/11329/2nd-ima-conference-on-inverse-problems-from-theory-to-application/ |
| Description | IMAGES network |
| Form Of Engagement Activity | Engagement focused website, blog or social media channel |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Other audiences |
| Results and Impact | Cambridge is home to a wealth of scientific research which includes developing tools for acquisition, visualization, processing and analysis of images. Mathematicians and engineers create image-processing and analysis algorithms, e.g. image reconstruction, image de-noising and restoration algorithms, methods for object segmentation and tracking, in particular in the presence of large-scale imaging data. At the MRRC, fluid-gas dynamics and chemical reactions are investigated using Magnetic Resonance Tomographic Imaging (MRI) as a visualisation and quantification technique. At the Cancer Research UK Cambridge Institute, light microscopy and medical imaging are exciting research fields. A collaboration between microscopists and mathematicians e.g. leads to new ways of tracking and analyzing the effect of cancer drugs during mitosis. At the FM/HKI spectroscopy methods underpin the non-invasive analyses of materials and techniques, conservation and cross-disciplinary interpretation of paintings, illuminated manuscripts and Egyptian papyri. The complex process, from acquiring images to their interpretation and problem-solving applications, requires multi-expertise partnerships. Different problems and image applications inform similar methodologies and interpretative strategies. Cross-disciplinary collaboration is needed to analyze the image-information not explicit in machine-generated data. For instance, in Magnetic Resonance Tomography, Positron Emission Tomography, microscopy imaging or seismic imaging, the combined expertise of mathematicians, engineers, computer scientists, medical doctors and geophysicists turns samples of Fourier-, Radon-transform data or electromagnetic waves into meaningful images of the brain, heart activity or ozone density in the atmosphere. Analyses of artist's materials and techniques, on the other hand, unite chemists, physicists, mathematicians, biologists, imaging scientists, conservators, artists and intellectual historians. We aim to stimulate new inquiries and focused dialogues between the sciences, arts and humanities by providing them with a platform for communication. |
| Year(s) Of Engagement Activity | 2014,2015,2016 |
| URL | http://www.images.group.cam.ac.uk |
| Description | IMAGiNG & MATHEMATiCS |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | Local |
| Primary Audience | Other audiences |
| Results and Impact | A workshop and open forum for scientists to discuss imaging problems and solutions. |
| Year(s) Of Engagement Activity | 2013,2014 |
| URL | http://www.damtp.cam.ac.uk/user/cbs31/Research_files/imaging%26mathematicsDec2013.pdf |
| Description | LMS meetings on Current frontiers in inverse problems: from theory to applications |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Other audiences |
| Results and Impact | In 1976 Keller formulated the following very general definition of inverse problems, which is often cited in the literature: "We call two problems inverses of one another if the formulation of each involves all or part of the solution of the other. Often, for historical reasons, one of the two problems has been studied extensively for some time, while the other is newer and not so well understood. In such cases, the former problem is called the direct problem, while the latter is called the inverse problem." Inverse problems appear in many situations in physics, engineering, biology and medicine. The main mathematical problem is the well (ill) - posedness of the inversion process. Indeed, in practice most inverse problems are ill-posed in terms of non-uniqueness or lack of stability of the inversion. We hold four LMS meetings on inverse problems every year that bring together researchers who work on advancing the field of inverse problems, both from a theoretical and from an applied point of view. |
| Year(s) Of Engagement Activity | 2014,2015,2016 |
| URL | http://www.damtp.cam.ac.uk/user/cbs31/LMS_Inverse_Day_Edinburgh/Home.html |
| Description | MFO mini-workshop on Deep Learning and Inverse Problems, 4-10 March 2018, Oberwolfach, Germany. Co-organisers: S. Arridge, M. de Hoop and P. Maass. |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Other audiences |
| Results and Impact | Oberwolfach Workshop on Inverse Problems and deep learning. |
| Year(s) Of Engagement Activity | 2018 |
| URL | https://www.mfo.de/occasion/1810c/www_view |
| Description | Mathematics and Information |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | Local |
| Primary Audience | Other audiences |
| Results and Impact | The Mathematics & Information Day is a half-day meeting, centred upon Cambridge, of researchers who are interested in the broad range of modern mathematical methodologies in understanding information-rich phenomena, e.g. data analysis, image and signal processing, medical imaging and compressed sensing. Slides of talks are online at http://www.damtp.cam.ac.uk/user/cbs31/MI_Cambridge/MathAndInfo_Network.html |
| Year(s) Of Engagement Activity | 2012 |
| URL | http://talks.cam.ac.uk/show/index/35381 |
| Description | Model-based learning in imaging |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | SIAM's Annual Meeting provides a broad view of the state of the art in applied mathematics, computational science, and their applications through invited presentation, prize lectures, minisymposia, contributed papers and posters. |
| Year(s) Of Engagement Activity | 2017 |
| URL | https://archive.siam.org/meetings/an17/ |
| Description | PLUS article What the eye can't see |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Public/other audiences |
| Results and Impact | Pictures play a vital role in our lives. They allow us to discover the world, to understand it and to enjoy it. Our own pair of eyes is a powerful tool, but modern imaging technology goes a lot further, revealing distant galaxies and tiny cells in our bodies. Mathematics is the language that underlies this technology, which is why the Cambridge Image Analysis Group, led by Carola-Bibiane Schönlieb, is at home in the Department of Applied Mathematics and Theoretical Physics at the University of Cambridge. |
| Year(s) Of Engagement Activity | 2016 |
| URL | https://plus.maths.org/content/what-eye-cant-see |
| Description | PLUS magazine Mathematical Moments |
| Form Of Engagement Activity | A press release, press conference or response to a media enquiry/interview |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Public/other audiences |
| Results and Impact | Mathematical moments: Carola-Bibiane Schönlieb Carola-Bibiane Schönlieb has a fascinating job: she works on the mathematics behind image analysis. It finds application in all sorts of areas, from medical imaging, such as MRI scans, to forest ecology, which sees scientists trying to gain information about forests from pictures taken from the air. In this brief interview Carola tells us why she likes doing maths, recalls some of her favourite mathematical moments, and explains why creativity is essential in mathematics. |
| Year(s) Of Engagement Activity | 2016 |
| URL | https://plus.maths.org/content/mathematical-moments-carola-schonlieb |
| Description | Re-visioning Transport and Health 2019, Workshop and Hackathon |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | In this workshop and hackathon, we assessed the state of the field and discussed how developments can be made to provide the data decision-makers in addressing the questions of health and sustainability for transport systems and the built environment. The workshop brought together experts in computer vision, earth observation, street-level data collection, population health, cities, and transport studies. The hackathon had two streams. The first stream involved improving/ developing algorithms to solve an image recognition problem and the second required interdisciplinary work to propose how to solve a real-world problem in a lower and middle income country with imaging data. |
| Year(s) Of Engagement Activity | 2019 |
| URL | https://sites.google.com/view/transportcam2019/home/workshop |
| Description | Seeing more in pictures - open mathematics day for Y12 girls |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Schools |
| Results and Impact | Dr Carola Schönlieb gives an insight into some of the mathematics behind image analysis and its wide-ranging applications in fields ranging from developing cancer therapies to restoring artworks, together with some personal reflections on her own career journey through mathematical study and research. Dr Carola Schönlieb is Reader in Applied and Computational Analysis and Head of Cambridge Image Analysis Group at the Department of Applied Mathematics and Theoretical Physics, University of Cambridge. This talk was originally given to an audience of Y12 girls (aged 16-17) at an event for students considering applying to university to study mathematics. The talk was recorded at the Centre for Mathematical Sciences, University of Cambridge, on 18 April 2016. |
| Year(s) Of Engagement Activity | 2016 |
| URL | https://www.youtube.com/watch?v=9SPN9Ouxx7g&feature=youtu.be |
| Description | Variational models and partial differential equations for mathematical imaging |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Plenary lecture at the Conference on the Numerical Solution of Differential and Differential-Algebraic Equations (NUMDIFF-15). Variational models and partial differential equations for mathematical imaging: Images are a rich source of beautiful mathematical formalism and analysis. Associated mathematical problems arise in functional and non-smooth analysis, the theory and numerical analysis of partial differential equations, harmonic, stochastic and statistical analysis, and optimisation. Starting with a discussion on the intrinsic structure of images and their mathematical representation, in this talk we will learn about variational models for image analysis and their connection to partial differential equations, and go all the way to the challenges of their mathematical analysis as well as the hurdles for solving these - typically non-smooth - models computationally. The talk is furnished with applications of the introduced models to image de-noising, motion estimation and segmentation, as well as their use in biomedical image reconstruction such as it appears in magnetic resonance imaging. |
| Year(s) Of Engagement Activity | 2018 |
| URL | https://sim.mathematik.uni-halle.de/numdiff/Numdiff15/index.html |
| Description | Winter School: The Mathematics of Imaging 7-11 January, 2019 |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Postgraduate students |
| Results and Impact | Helped to co-organise this Winter school "The mathematics of Imaging", that was held in Paris at the IHP (Institut Henri Poincaré), from January 7 to April 5, 2019. The event included course, practical sessions, flash presentations and posters which created much discussion and hopefully new collaborations. |
| Year(s) Of Engagement Activity | 2019 |
| URL | https://imaging-in-paris.github.io/semester2019/school/ |
| Description | Workshop on Gradient flows: challenges and new directions, ICMS 2018 |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Postgraduate students |
| Results and Impact | Gradient flows: challenges and new directions ICMS, The Bayes Centre, 47 Potterrow, Edinburgh EH8 9BT 10 - 14 September 2018 ORGANISERS Bertram Düring, University of Sussex Carola-Bibiane Schönlieb, University of Cambridge Yves van Gennip, University of Nottingham Marie-Therese Wolfram, University of Warwick |
| Year(s) Of Engagement Activity | 2018 |
| URL | https://www.icms.org.uk/gradientflows.php |
| Description | Workshop on Statistics, Learning and Variational Methods in Imaging |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Other audiences |
| Results and Impact | Scope: We bring together young researchers with a variational background, statisticians and experts in learning theory to work on the following challenging topics: - statistics of noise and its modelling - modelling a-priori information - learning the above from samples - Bayesian and variational approaches The workshop is setup as a combination of presentations from invited speakers and discussion rounds to drive an exchange between the participants from different research areas involved. |
| Year(s) Of Engagement Activity | 2012 |
| URL | http://www.damtp.cam.ac.uk/user/cbs31/Imaging_Workshop_Cambridge_September_2012/Workshop.html |