Stochastic iterative regularization: theory, algorithms and applications

An inverse problem arises whenever one seeks the cause of observed physical phenomena or observational data, e.g., inferring the governing law from the measurements. This task essentially underlies all scientific discoveries and technological innovations. Thus, the mathematical theory and computational techniques for solving inverse problems are central, e.g., in physics, astronomy, medicine, engineering, and life sciences, and it has evolved into a highly interdisciplinary research area.

Inverse problems are usually ill-posed in the sense that the sought-for solution lacks existence, uniqueness or stability with respect to data perturbation. Since the noise is inherent in the observational data, the numerical algorithms have to employ specialized techniques, commonly known as regularization. The corresponding mathematical framework in the form of regularization theory is highly developed, since the pioneering works of A. Tikhonov in 1960s, H. Engl et al from 1980s and many other researchers. This theory has played a vital role in many research areas, and related numerical algorithms have also been intensively investigated. One versatile framework is to minimize an objective function measuring the quality of fitting between the model output and observational data, possibly plus some additional penalty term, and it covers a large class of powerful iterative inversion techniques.

Due to the unprecedented advances in data acquisition technologies, large datasets are becoming common place for many practical inverse problems. Prominent examples in medical imaging include dynamic, multispectral, multi-energy or multi-frequency data in computed tomography and optical tomography. The ever increasing volume of available data poses enormous computational challenges to image reconstruction, and traditional iterative methods can be too expensive to apply, and currently it represents one of the bottlenecks to extract useful information from the massive dataset. This is especially challenging for problems involving complex physical models, where each data set is very expensive to simulate.

The proposed research aims at addressing the aforementioned outstanding computational challenge using stochastic iterative techniques developed within the machine learning community, and providing relevant theoretical underpinnings. The central idea of stochastic iterative methods is that at each step only a (small) portion of the data set is used to steer the progression of the iterates, instead of the full data set. This allows drastically reducing the computational cost per iteration. This idea has received enormous attention within the machine learning community, and especially has achieved stunning success in deep learning in recent years. Actually stochastic gradient descent and its variants are the workhorse behind many deep learning tasks. A successful completion of this project will greatly advance modern image reconstruction by providing a systematic mathematical and computational framework, including comprehensive theoretical underpinnings, novel algorithms and detailed studies on concrete inverse problems, e.g., in medical imaging.

The main beneficiaries of the outcomes of this project in terms of theoretical developments and computational techniques would be research institutes or public sectors where the inverse theory and algorithms play a crucial role, e.g., various research centres on inverse problems, signal / image processing and data sciences, e.g., Centre for Inverse Problems at UCL (joint between computer science, mathematics and statistics), Centre for Image Analysis at Cambridge University, AI Centre at UCL Computer Science, and Alan Turing Institute (UCL is one of the founding partner institutes) in the UK, and Radon Institute of Applied and Computational Mathematics in Austria and Centre of Excellence for Inverse Modelling and Imaging in Finland. To disseminate the research outputs to the community, we will give presentations regularly at leading conferences, e.g., Applied Inverse Problems conference (2019, 2021), SIAM Conference on Imaging Sciences (2020, 2022), Chemnitz Symposium in Inverse Problems in Germany and Inverse Days in Finland, and the conference trips will be combined with research visits and seminar talks whenever appropriate to maximize the potential impact.

The main beneficiaries of the algorithmic outcomes and concrete inverse problems in medical imaging within the project would be research institutes or public sections where reconstruction technologies are heavily used, e.g., Centre for Medical Image Computing at UCL (jointly between computer science and medical physics at UCL), AI Centre at UCL and Alan Turing Institute in the UK. Through the connections between Department of Computer Science and these institutes, there will be many opportunities to collaborate directly, e.g., experimental evaluation of the developed technologies. These connections ensure a direct conduit of the transfer of the obtained technologies to medical imaging and machine learning. This transfer will be greatly facilitated through the software package to be developed, as well as the proposed research on novel applications in optical tomography etc.

The wider public will benefit in many ways from a successful completion of this project, e.g., in terms of a significant improvement in medical imaging quality and speed (hence more accurate and faster diagnosis of diseases) achieved by the new algorithms. Thus it will have significant repercussions on all economic and societal sectors involved. The outputs can potentially attract companies, e.g., on imaging device manufacturers (e.g., Phillips, GE and Siemens), and on data science and image processing companies (e.g., Microsoft Research and DeepMind). To achieve this transfer, we will regularly attend industry-sponsored events to disseminate relevant research results, e.g., the activities at Alan Turing Institute.