# Coupling and Control in Continuous Time

Lead Research Organisation: King's College London
Department Name: Mathematics

### Abstract

Randomness is ubiquitous in the natural world, and advances in understanding and modelling random events are key to making progress with many problems in the natural and social sciences, engineering, statistics, to name but a few. Coupling is a fundamental paradigm in probability through which probability distributions of random quantities (random variables, random processes) can be compared with each other via "pointwise" comparisons. It yields powerful techniques for analysing random systems.

A Markov process is a random process whereby, conditional on the present, its future and past are independent. That is, if we know the present state of the process, we can gain no additional information on its future evolution by knowing more about its past. This paradigm describes many random processes used as models in the natural and social sciences. In coupling we are looking at two Markov processes that start from different locations and evolve jointly. We are interested in them meeting a number of criteria, e.g. the two processes meeting as soon as possible, staying close to each other for as long as possible, or other criteria (e.g. the large deviation behaviour of the coupling time, i.e. what the exponential rate of decay of the coupling time is). As well as being an interesting mathematical question in and of itself, this problem has significant potential applications. For example, the rate of convergence to stochastic equilibrium (a crucial question in many applications) is controlled by the rate at which coupling occurs.

There is a natural lower bound in the speed of coupling. The "fastest" couplings, i.e. the couplings where the probability that the two processes have not met by any given time is smallest, are known as "maximal" couplings: one can construct those by defining the second process as a functional of the entire trajectory of the first. However, in the context of modelling in the sciences, it is natural to focus on co-adapted couplings, namely couplings whereby the second process at a given time can only be constructed based on the trajectory of the first upto and including the present time (i.e. no information about the future trajectory of the first process can be taken into account).

The difficulty here is that it is hard to obtain optimal (called "extremal") couplings. In fact it's difficult to know how good any given co-adapted coupling is. This proposal is about taking any co-adapted coupling and providing a method of improving it. Not just locally, but proving mathematically that the sequential improvements we propose yield a co-adapted coupling that is as good as it can get. Essentially we are looking to solve a stochastic optimisation problem under the additional constraint of co-adaptivity.

In this proposal, the main method for improving a co-adapted coupling to achieve optimality is via the application of control theory. We aim to use the Policy Improvement Algorithm, a tool from control theory that works in discrete time, and develop its application in continuous time. In the application part of the project, we aim to develop applications of the PIA in the theory of non-linear PDEs and Multi-Level Monte Carlo (MLMC) algorithms for processes with jumps. The areas of non-linear PDEs and MLMC simulation have applications with vast societal and economic impact: the former has applications in biology, physics, engineering to name a few, and the latter is of crucial importance in Uncertainty Quantification in engineering and science. When the uncertainty is high-dimensional and strongly nonlinear, Monte Carlo simulation remains the preferred approach, with applications in areas as diverse as biochemical reactions and plasma physics.

### Planned Impact

Societal and economic impact. The Coupling and Control methods to be developed in the project have significant applications in the areas of non-linear PDEs and multi-level Monte Carlo (MLMC) simulation. Both of these areas have vast societal and economic impact: the former has applications in biology, physics, engineering to name a few, and the latter is of crucial importance in "Uncertainty Quantification" in engineering and science. Within non-linear PDEs, as an example, the ability to solve quickly the multi-dimensional Fisher-Kolmogorov (also known as FKPP) equation to find multiple genes would be of interest to epidemiologists and geneticists. There are also possible impacts in engineering: some of the results of the project may lead to algorithms for stochastic control in real time. When the uncertainty is high-dimensional and strongly nonlinear, Monte Carlo simulation remains the preferred approach, with applications in areas as diverse as biochemical reactions and plasma physics.

Human Resources. The project meets the national need of training homegrown researchers in probability, identified by the International Review of Mathematics 2010. The two young researchers who will be devoted to the project, the RCoI Dr Matija Vidmar and the PDRA Mr Jure Vogrinc, will receive world-class research training, not only within probability theory, but also in collaboration with applied mathematics. In our view such cross-disciplinary work contributes to the health of both research disciplines involved and plays an increasingly important role in the development of science.

The young researchers will receive full support to attend the largest international conferences in the field and to present their own work. They will also be encouraged to visit other teams, develop international contacts and invite their own visitors with the objective of building their own network of collaborations. They will particularly benefit from exposure to and collaboration with the world leading experts who are keen to contribute to the project: Professors Aldous, Burdzy and Thorisson, who are world leaders in the area of Coupling, and Professors Karatzas, Krylov, Pham and Soner who are leaders in Stochastic Control and Stochastic Numerical Methods for PDEs. In terms of general aspects of staff development and support, well-resourced programmes are offered by KCL and Warwick. Both PIs have a long track-record of PhD and postdoctoral advising. Their former PhD students and postdocs have found positions in academia or industry. Researchers with strong experience in mathematical and numerical modeling are in high demand in the UK and across the world. The young researchers hired on the project will subsequently be in a very strong position to compete for highly skilled jobs, be they in academia or in industry.

### Publications

10 25 50
Georgiou N (2018) A radial invariance principle for non-homogeneous random walks in Electronic Communications in Probability

Georgiou N (2019) Invariance principle for non-homogeneous random walks in Electronic Journal of Probability

Georgiou Nicholas (2018) Invariance principle for non-homogeneous random walks in arXiv e-prints

González Cázares J (2019) Exact simulation of the extrema of stable processes in Advances in Applied Probability

González Cázares J (2020) $\varepsilon$-strong simulation of the convex minorants of stable processes and meanders in Electronic Journal of Probability

Jacka Saul (2017) Multi-currency reserving for coherent risk measures in arXiv e-prints

### Related Projects

Project Reference Relationship Related To Start End Award Value
EP/P003818/1 01/11/2016 31/07/2018 £329,788
EP/P003818/2 Transfer EP/P003818/1 01/08/2018 31/10/2019 £159,934

Description We made substantial progress in four main areas.

1. Develop the Policy Improvement Algorithm (PIA) and explore its convergence properties.

We have demonstrated incredibly fast (quadratic - so the next error is the square of the previous one) convergence of the PIA - which goes a long way to explain its rapid convergence in general settings.
This result is conditional on having a convergence result in the first place -we provide this in a general diffusion setting. The result critically depended on a novel coupling construction, generalising the coupling of Rogers and Lindvall.

2. Apply and optimise coupling methods

We've found innovative applications of coupling in diverse settings in the theory of convergence and stability of continuous and discrete time stochastic processes with and without jumps: apart from Multi-Level Monte Carlo (MLMC), we have given applications to stability of the Stochastic Gradient Langevin Algorithm, which is very widely used in Machine Learning. Before this work, which is based on a new coupling construction, the theory was only applicable to a limited class of potentials.

3. Develop control of semimartingales with jumps related PIDEs paper

We have made a substantial leap in the theory of control of continuous time stochastic processes with jumps-and are starting to explore the applications to solutions to nonlinear PIDEs. Our findings will be pivotal in the workshop we are organising on Optimal Control in Fractional Dynamics during the 2021 Isaac Newton Institute programme on Fractional Differential Equations.

4. Devise and refine Monte Carlo methods for processes with jumps

We have designed a new coupling between Levy processes which yields a novel characterisation of the extrema of a Levy process. The characterisation is essential in the construction of a new simulation algorithm. It allows for the types of couplings which are needed for MLMC, providing a surprisingly efficient way of simulating various essential attributes of the process.
Exploitation Route Innovations in stochastic control of jump processes will be of substantial relevance to specialists in the application of control.
Machine Learning is applied everywhere. The innovations from the project, particularly those related to the Stochastic Langevin Dynamics Algorithm will be applied widely in the training phase for ML algorithms.
Sectors Digital/Communication/Information Technologies (including Software),Education,Energy,Financial Services, and Management Consultancy,Security and Diplomacy

Description 2017 UK-Mexico Visiting Chair Mobility Grants
Amount £6,000 (GBP)
Organisation King's College London
Country United Kingdom
Start 06/2018
End 07/2018

Description DMS-EPSRC: Fast martingales, large deviations and randomised gradients for heavy-tailed target distributions
Amount £662,611 (GBP)
Funding ID EP/V009478/1
Organisation Engineering and Physical Sciences Research Council (EPSRC)
Sector Public
Country United Kingdom
Start 03/2021
End 09/2024

Description Defence and Security Programme at the Alan Turing Institute
Amount £55,000 (GBP)
Sector Public
Country United Kingdom
Start 11/2017
End 03/2018

Description Fellowship extension
Amount £10,000 (GBP)
Funding ID ARC-1498-18-19-AP
Organisation University of Warwick
Country United Kingdom
Start 11/2019
End 05/2020

Description Lloyds Register Foundation Programme on Data Centric Engineering
Amount £100,000 (GBP)
Organisation Lloyd's Register Foundation
Sector Charity/Non Profit
Country United Kingdom
Start 10/2017
End 09/2020

Description Policy improvement, jump control and reinforcement learning
Amount £32,530 (GBP)
Organisation Alan Turing Institute
Country United Kingdom
Start 04/2020
End 09/2020

Description Prediction of illiquid corporate bond prices using deep learning & signature methods
Country United Kingdom
Sector Private
PI Contribution Development of novel methods for price prediction of illiquid securities using an approach based on rough path theory and machine learning
Collaborator Contribution BlackRock have contributed insight into the problem, market knowledge and relevant data.
Impact Algorithm based on rough paths methods for the analysis and prediction of illiquid bond prices.
Start Year 2018

Description Royal Society Discussion Meeting "How should pension liabilities be valued? Risk aversion and demographic uncertainty."
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact 80+ finance and pensions practitioners together with academic experts attended a Royal Society Discussion Meeting "How should pension liabilities be valued? Risk aversion and demographic uncertainty."
The purpose was to explore and influence attitudes to risk-monetary, demographic and investment, in relation to the valuation and governance of defined benefit pension schemes.
Year(s) Of Engagement Activity 2019
URL https://royalsociety.org/science-events-and-lectures/2019/03/pensions/