Machine learning and dynamical systems

This project looks to deep learning to predict and learn non-autonomous dynamical systems with known equivariances. While there exists a formal definition for the equivariance of a system, equivariant systems can be and are commonly characterised by their symmetries. Dynamical systems of this nature have been the object of attention in the mathematical physics community for a very substantial period of time and, while the idea of learning non-integrable dynamical systems is hardly new, research into using machine learning algorithms to do so is only-just emerging and evolving with the field itself. Neural networks are not traditionally seen as a natural setting to learn to predict continuous-time systems, but research and observation over the past several years has found connections between the problem of training neural networks and a standard problem in optimal control theory. At the same time, convolutional neural networks have been found to enforce equivariance of the function that they learn and represent. It is precisely these connections that inspire this current project.

This project adds to this research with the broader aim of integrating the significant, existing theory of dynamical systems into a deep learning framework using neural networks. If full advantage of such material is taken and used to inform current machine learning approaches, significant improvements in results are expected; systematic exploration of the intersection of these fields is both recent and limited in that most research falls short in providing sufficient rigorous evidence to support its chosen methodology.

As such, it is the intention of this project to build a rigorous and full-bodied approach to learning such systems, which should include the following: analytical statements of the accuracy and error of methods as well as their reliability; practical implementations of methods and the efficiency and time-accuracy trade-offs that they bring; and, most importantly, a clear understanding of how properties of and results for such systems (stemming from but not limited to algebra, differential geometry, group theory and geometric mechanics) can be leveraged and best applied to every stage of the machine learning process (the selection of the model, architecture, learning principle, optimisation etc). Such understanding in the field is mainly empirical and speculative in nature, rarely supported by clear mathematical proof. For machine learning algorithms of this kind to be used and applied in real-world examples as hoped for, it is of the utmost importance that proof of the ability to accurately learn and predict such systems can be given explicitly.

Classical, integrable systems in a low dimension with known solutions and extensive previous study provide an appropriate stepping-off point for the project; as a result, at first, a restriction to learning Hamiltonian systems is made, with the intention of extending and building on the approach for non-integrable and/or high-dimensional systems. Initial problems that are looked at for Hamiltonian systems not only include learning their dynamics but the underlying variational principle and corresponding symmetry itself that drive the system, with the hope of generalising this to systems where an unknown variational principle is suspected to be at play.

Additional potential areas of application (other than systems of interest to the mathematical physics community) range from data describing complex non-linear systems (for example, molecular data) to image classification.

This project falls within the following EPSRC research areas: artificial intelligence technologies; mathematical physics; mathematical analysis; non-linear systems.

Probabilistic modelling permeates the Financial services, healthcare, technology and other Service industries crucial to the UK's continuing social and economic prosperity, which are major users of stochastic algorithms for data analysis, simulation, systems design and optimisation. There is a major and growing skills shortage of experts in this area, and the success of the UK in addressing this shortage in cross-disciplinary research and industry expertise in computing, analytics and finance will directly impact the international competitiveness of UK companies and the quality of services delivered by government institutions.
By training highly skilled experts equipped to build, analyse and deploy probabilistic models, the CDT in Mathematics of Random Systems will contribute to
- sharpening the UK's research lead in this area and
- meeting the needs of industry across the technology, finance, government and healthcare sectors

MATHEMATICS, THEORETICAL PHYSICS and MATHEMATICAL BIOLOGY

The explosion of novel research areas in stochastic analysis requires the training of young researchers capable of facing the new scientific challenges and maintaining the UK's lead in this area. The partners are at the forefront of many recent developments and ideally positioned to successfully train the next generation of UK scientists for tackling these exciting challenges.
The theory of regularity structures, pioneered by Hairer (Imperial), has generated a ground-breaking approach to singular stochastic partial differential equations (SPDEs) and opened the way to solve longstanding problems in physics of random interface growth and quantum field theory, spearheaded by Hairer's group at Imperial. The theory of rough paths, initiated by TJ Lyons (Oxford), is undergoing a renewal spurred by applications in Data Science and systems control, led by the Oxford group in conjunction with Cass (Imperial). Pathwise methods and infinite dimensional methods in stochastic analysis with applications to robust modelling in finance and control have been developed by both groups.
Applications of probabilistic modelling in population genetics, mathematical ecology and precision healthcare, are active areas in which our groups have recognized expertise.

FINANCIAL SERVICES and GOVERNMENT

The large-scale computerisation of financial markets and retail finance and the advent of massive financial data sets are radically changing the landscape of financial services, requiring new profiles of experts with strong analytical and computing skills as well as familiarity with Big Data analysis and data-driven modelling, not matched by current MSc and PhD programs. Financial regulators (Bank of England, FCA, ECB) are investing in analytics and modelling to face this challenge. We will develop a novel training and research agenda adapted to these needs by leveraging the considerable expertise of our teams in quantitative modelling in finance and our extensive experience in partnerships with the financial institutions and regulators.

DATA SCIENCE:

Probabilistic algorithms, such as Stochastic gradient descent and Monte Carlo Tree Search, underlie the impressive achievements of Deep Learning methods. Stochastic control provides the theoretical framework for understanding and designing Reinforcement Learning algorithms. Deeper understanding of these algorithms can pave the way to designing improved algorithms with higher predictability and 'explainable' results, crucial for applications.
We will train experts who can blend a deeper understanding of algorithms with knowledge of the application at hand to go beyond pure data analysis and develop data-driven models and decision aid tools
There is a high demand for such expertise in technology, healthcare and finance sectors and great enthusiasm from our industry partners. Knowledge transfer will be enhanced through internships, co-funded studentships and paths to entrepreneurs