Sparsity Regularization for Inverse Problems -- Theory, Algorithm and Application

One fundamental task in many scientific and engineering disciplines is to probe the world around us, often in the form of deducing physical laws or determining the parameters therein from experimental data. This gives rise to a wide variety of inverse problems of inferring the cause of observed physical phenomena or determining medium properties of the concerned object from the measured responses. A model based approach to inverse problems assumes a known relation between the unknown physical quantity and the measured responses, in the form of linear or nonlinear equations. Inverse problems are numerically challenging to solve since they are unstable with respect to data noise. One of the most successful and powerful techniques is to incorporate a priori knowledge by means of regularization. This project focuses on one specific regularization technique based on sparsity constraints.

In sparsity regularization, one looks for an approximate solution that has as few nonzero entries (or a small support) as possible. That is, we aim at explaining the physical phenomenon by simply using a small number of parameters. In this project, we study the mathematical theory, computational techniques and practical applications of the approach, especially by using a nonconvex penalty on the sought-for solution. There are several nonconvex penalties proposed in the literature, especially in statistics and machine learning, and we shall consider, for example, the popular l0 penalty, bridge penalty, smoothly clipped absolute deviation, minmax concavity penalty and clipped l1 penalties. Despite their popularity, their use and study in the context of inverse problems remain very limited. The proposed project examines these techniques in the framework of inverse problems theory. Specifically, we focus on the following objectives during the project period: (a) to develop a computational method of primal-dual active set type for efficiently solving the regularized model (with nonconvex penalties) and rigorously establish the convergence of the algorithm; (b) to develop applicable choice rules for the regularization parameter, and to analyze the structural properties of "local" minimizers; (3) to apply the nonconvex approach to tomography imaging, by combining it with an adaptive finite element method.

The computational technique to be developed, primal dual active set algorithm, is widely applicable to many other areas, especially machine learning and statistics, since these nonconvex models have their origins and motivations there. Mathematically, the theory of sparsity regularization will shed valuable lights into the analysis of nonsmooth regularization, which shares common structures with many important mathematical models arising in imaging and signal processing. The specific choice rule for the regularization parameter will enable automatic parameter selection yet with rigorous theoretical justification, and improve the efficiency of the current practice based on tedious trial and error. The research in tomography imaging, i.e., the application of nonconvex penalty and adaptive algorithm, will directly impact the medical imaging community. There are a large group of researchers working on tomography imaging that will benefit directly from the research, and the obtained research results will be presented to them continuously. More generally, it provides guidance for developing efficient algorithms for nonlinear inverse problems for differential equations. In summary, the project will take sparsity regularization for inverse problems to the next level.

Sparsity regularization is a highly active research area in Inverse Problems, and it has a broad range of applications in biomedical imaging, e.g., magnetic resonance imaging, and industrial process such as geophysical prospecting and remote sensing. The proposed research is likely to significantly expand the scope of inverse problems. The obtained research results will benefit not only researchers specialized in Inverse Problems, but also broader communities, e.g., machine learning, statistics, image and signal processing. Further, the research on the application of the sparsity regularization to tomography imaging will directly impact related medical imaging modalities and hence human well-being in the UK. We discuss these impacts separately below.

1) Sparsity regularization is an important example of nonsmooth regularization (or regularization in Banach spaces), which is widely applied in the broad area of imaging. The mathematical techniques developed might be extended to many other important imaging models, e.g., structural sparsity, leading to better reconstructions and facilitating the use of these models in practice.

2) The nonconvex models discussed in the project were originally proposed in statistical literature and machine learning, and have gained immense interest in statistics. These models are expected to play an important role in analyzing and learning "big data", e.g., the recommendation system of popular Amazon portal and film evaluation at the online movie provider NetFlix, where sparsity represents an important feature. The computational techniques developed will help solve related mathematical models efficiently, and thereby the research will improve the quality of daily life, in addition to promoting the research activities in statistics and machine learning.

3) The software package to the released is applicable to many different model, and potentially adopted in commercial packages (e.g., specialized in medical imaging). Experience indicates that such a practice makes an enormous difference to the impact of new computational methods, because it allows users to test and evaluate the techniques to an unlimited extent. Feedback in terms of bug reports, discussions, and contributed improvements can take place much faster than through formal publication channels, and will lead to strong evidence for confidence via extensive independent testing.

4) In the project, we shall study the application of sparsity regularization to tomography imaging, such as electrical impedance tomography and diffuse optical tomography, by combining it with the adaptive finite element method. The research in this aspect will directly impact medical imaging techniques, either enhancing the reconstruction resolution or improving the computational efficiency. The Center for Medical Image Computing at UCL collaborates intensively with faculties from Department of Medical Physics and Bioengineering, which provides a seamless conduit for transferring latest mathematical developments into practical applications. We expect the techniques we develop to be taken up in several other applications both within biomedical imaging and further, in non-destructive testing, industrial tomography, and in geophysical applications including oil and gas exploration.

We expect that with these developments, the research field of "nonconvex sparsity regularization" is likely to provoke intense interest from mathematicians, physicists, computational scientists, statisticians, and engineers.