Man, Machine, and Mathematics: A mathe-physical search for differential equation solutions via deep learning

Well posed mathematical problems are seldom solvable in an explicit, conventional sense. A notable example of this phenomenon is within the field of differential equations, where an exact solution often exists but is analytically impossible to find. The development of approximation methods to estimate such well-parametrized but not explicitly knowable solutions is an important endeavour within the field.

This project presents a blueprint for building such approximations via deep learning. We build our methods on assumptions satisfied by a large class of differential equations, such as Frechet differentiability of the equation operators and compact domains of interest. We aim to demonstrate that neural network solution models are almost always capable of being found under such assumptions. We also hope to present explicit results on the errors expected from these models, alongside techniques for quantifying and minimising those errors. Finally, we aim to provide strict guarantees on the model sizes, architectures, optimization run-times, etc., needed to search for models that are accurate up to pre-specified tolerances.

The project will hope to advance the field of neural network-based differential equation solving by adding two novel facets to it. First, by fusing ideas from numerical methods and error analysis into deep learning of parametrized objects, we hope to move the field away from its reliance on surrogate markers, like loss functions, as a means of error analysis. In turn, that will also lead us to methods of efficient error correction.

Second, we aim to provide a rigorous set of a priori-decidable strategies for efficient model building by leveraging the emerging advances in computing infrastructure and the algorithms that can tap into them.

Indeed, the project has already seen its first successes in the modelling of dynamical systems' differential equations (Physical Review E 105, 065305) and in sparsifying deep networks (Redman et al., ICML 2022). Given the generality of the results developed in the latter work (it concerns all deep networks, not just differential equation solution models), we believe the project will lead to significant results beyond its originally planned scope.

Success in these objectives will require the creation of new techniques within the fields of random dynamical systems, stochastic computational methods, deep learning methods for equation solving, and optimization/complexity analysis, amongst others, each of which plays a central role in the CDT's research and training missions.

Probabilistic modelling permeates the Financial services, healthcare, technology and other Service industries crucial to the UK's continuing social and economic prosperity, which are major users of stochastic algorithms for data analysis, simulation, systems design and optimisation. There is a major and growing skills shortage of experts in this area, and the success of the UK in addressing this shortage in cross-disciplinary research and industry expertise in computing, analytics and finance will directly impact the international competitiveness of UK companies and the quality of services delivered by government institutions.
By training highly skilled experts equipped to build, analyse and deploy probabilistic models, the CDT in Mathematics of Random Systems will contribute to
- sharpening the UK's research lead in this area and
- meeting the needs of industry across the technology, finance, government and healthcare sectors

MATHEMATICS, THEORETICAL PHYSICS and MATHEMATICAL BIOLOGY

The explosion of novel research areas in stochastic analysis requires the training of young researchers capable of facing the new scientific challenges and maintaining the UK's lead in this area. The partners are at the forefront of many recent developments and ideally positioned to successfully train the next generation of UK scientists for tackling these exciting challenges.
The theory of regularity structures, pioneered by Hairer (Imperial), has generated a ground-breaking approach to singular stochastic partial differential equations (SPDEs) and opened the way to solve longstanding problems in physics of random interface growth and quantum field theory, spearheaded by Hairer's group at Imperial. The theory of rough paths, initiated by TJ Lyons (Oxford), is undergoing a renewal spurred by applications in Data Science and systems control, led by the Oxford group in conjunction with Cass (Imperial). Pathwise methods and infinite dimensional methods in stochastic analysis with applications to robust modelling in finance and control have been developed by both groups.
Applications of probabilistic modelling in population genetics, mathematical ecology and precision healthcare, are active areas in which our groups have recognized expertise.

FINANCIAL SERVICES and GOVERNMENT

The large-scale computerisation of financial markets and retail finance and the advent of massive financial data sets are radically changing the landscape of financial services, requiring new profiles of experts with strong analytical and computing skills as well as familiarity with Big Data analysis and data-driven modelling, not matched by current MSc and PhD programs. Financial regulators (Bank of England, FCA, ECB) are investing in analytics and modelling to face this challenge. We will develop a novel training and research agenda adapted to these needs by leveraging the considerable expertise of our teams in quantitative modelling in finance and our extensive experience in partnerships with the financial institutions and regulators.

DATA SCIENCE:

Probabilistic algorithms, such as Stochastic gradient descent and Monte Carlo Tree Search, underlie the impressive achievements of Deep Learning methods. Stochastic control provides the theoretical framework for understanding and designing Reinforcement Learning algorithms. Deeper understanding of these algorithms can pave the way to designing improved algorithms with higher predictability and 'explainable' results, crucial for applications.
We will train experts who can blend a deeper understanding of algorithms with knowledge of the application at hand to go beyond pure data analysis and develop data-driven models and decision aid tools
There is a high demand for such expertise in technology, healthcare and finance sectors and great enthusiasm from our industry partners. Knowledge transfer will be enhanced through internships, co-funded studentships and paths to entrepreneurs