Modern Linear Algebra for PDE-Constrained Optimisation Models for Huge-Scale Data Analysis

What accurately describes such real-world processes as fluid flow mechanisms, or chemical reactions for the manufacture of industrial products? What mathematical formalism enables practitioners to guarantee a specific physical behaviour or motion of a fluid, or to maximise the yield of a particular substance? The answer lies in the important scientific field of PDE-constrained optimisation.

PDEs are mathematical tools called partial differential equations. They enable us to model and predict the behaviour of a wide range of real-world physical systems. From the optimisation point-of-view, a particularly important set of such problems are those in which the dynamics may be controlled in some desirable way, for instance by applying forces to a domain in which fluid flow takes place, or inserting chemical reactants at certain rates. By influencing a system in this way, we are able to generate an optimised outcome of a real-world process. It is hence essential to study and understand PDE-constrained optimisation problems.

The possibilities offered by such problems are immense, influencing groundbreaking research in applied mathematics, engineering, and the experimental sciences. Crucial real-world applications for such problems arise in fluid dynamics, chemical and biological mechanisms, weather forecasting, image processing including medical imaging, financial markets and option pricing, and many others. Although a great deal of theoretical work has been undertaken for such problems, it has only been in the past decade or so that a focus has been placed on solving them accurately and robustly on a computer, by tackling the matrix systems of equations which result. Much of the research underpinning this proposal involves constructing powerful iterative methods accelerated by 'preconditioners', which are built by approximating the relevant matrix in an accurate way, such that the preconditioner is much cheaper to apply than solving the matrix system itself. Applying our methodology can then open the door to scientific challenges which were previously out of reach, by only storing and working with matrices that are tiny compared to the systems being solved overall.

Recently, PDE-constrained optimisation problems have found crucial applicability to problems from data analysis. This is due to the vast computing power that is available today, meaning that there exists the potential to store and work with huge-scale datasets arising from commercial records, online news sites, or health databases, for example. In turn, this has led to a number of applications of data-driven processes being successfully modelled by optimisation problems constrained by PDEs. It is essential that algorithms for solving problems from these applications of data science can keep pace with the explosion of data which arises from real-world processes. Our novel numerical methods for solving the resulting huge-scale matrix systems aim to do exactly this.

In this project, we will examine PDE-constrained optimisation problems under the presence of uncertain data, image processing problems, bioinformatics applications, and deep learning processes. For each problem, we will devise state-of-the-art mathematical models to describe the process, for which we will then construct potent iterative solvers and preconditioners to tackle the resulting matrix systems. Our new algorithms will be validated theoretically and numerically, whereupon we will then release an open source code library to maximise their applicability and impact on modern optimisation and data science problems.

The economic and societal impact of the proposed research will primarily be realised through the development of accurate and efficient tools for PDE-constrained optimisation problems, as well as models and numerical methods to apply these formulations to real-world data analysis problems. The work packages outlined in this proposal, relating to PDE optimisation under the presence of uncertain data, image processing, bioinformatics, and deep learning, are of clear practical value for real-world mechanisms. We will collaborate and communicate extensively with industrial partners interested in applications of optimisation and data science, and thus provide clear benefit to a range of sectors whose work hinges on these crucial fields.

One primary route to impact for this project is through the production of high-quality numerical software, thus addressing a traditional bottleneck in applied mathematics research that the uptake of new algorithms by practitioners can take a considerable amount of time due to the lack of available code. We will work with the Numerical Algorithms Group, the Project Partner on this proposal and a leading provider of mathematical and statistical software to industry, to develop robust and usable algorithms and code for data analysis problems, thus guaranteeing an industry presence for our methodologies.

We will also carry out collaborative work with researchers at Google, on numerical methods for the bioinformatics and deep learning problems considered in this proposal. This expertise will enhance the quality and reach of the research to the industrial communities in machine learning and neural networks. We will interact with other relevant industries and agencies, such as NHS Scotland (relating in particular to medical imaging), Amazon, and the British Geological Survey. We will continuously aim to forge new links with further companies interested in our research, and will additionally reach out to interdisciplinary researchers at the University of Edinburgh (Schools of Engineering and Informatics, EPCC, medical imaging groups in the Edinburgh Medical School), other universities, the Alan Turing Institute, and their own industry contacts. We will create our own open source code library on PDE-constrained optimisation for data analysis which will be used by, and actively publicised to, further industry practitioners.

The proposal also includes the organisation of a 2-day workshop involving industrial and interdisciplinary partners, which will further stimulate interest in the application areas of this project, facilitate knowledge exchange to and from practitioners, and address open questions in industry.

Another key beneficiary of this proposal will be the PDRA themself, who will be provided with comprehensive training and supervision in computational optimisation, numerical solution of PDEs and PDE-constrained optimisation problems, applied mathematics, data science, and numerical software development. Along with the range of professional support and other opportunities available, this will ensure that the PDRA is strongly placed to take up a subsequent position in either academia or industry upon completion of the project, thus also aiding UK science in training highly-skilled mathematicians.

Both the PDRA and the Investigator will supervise MSc projects on the subject of this proposal, as well as related topics, thus also benefiting these research students. Finally, we will ensure substantial knowledge transfer through the publication of scientifically-leading research articles in top-tier international journals, the presentation of research at pre-eminent national and international conferences, and the delivery of additional seminars and lectures at UK and overseas institutions as well as to industrial partners.