Stochastic Processes and Partial Differential Equations for problems in Statistics and Machine learning
Lead Research Organisation:
Imperial College London
Department Name: Mathematics
Abstract
The project will investigate the application of continuous time stochastic processes for optimisation and sampling problems. The aim is to clarify links between optimisation and sampling and the long term behaviour of stochastic processes, such as accelerated or self interacting diffusions. We will consider the properties and performance of appropriate time discretisations when intended for contemporary applications from Statistics and Machine learning, such as learning parameters of deep neural networks, high dimensional regression or Bayesian variable selection.
Organisations
People |
ORCID iD |
| Grigorios Pavliotis (Primary Supervisor) | |
| Martin Chak (Student) |
Publications
Chak M
(2023)
Optimal friction matrix for underdamped Langevin sampling
in ESAIM: Mathematical Modelling and Numerical Analysis
Studentship Projects
| Project Reference | Relationship | Related To | Start | End | Student Name |
|---|---|---|---|---|---|
| EP/N509486/1 | 30/09/2016 | 30/03/2022 | |||
| 2129618 | Studentship | EP/N509486/1 | 30/09/2018 | 29/06/2022 | Martin Chak |
| EP/R513052/1 | 30/09/2018 | 29/09/2023 | |||
| 2129618 | Studentship | EP/R513052/1 | 30/09/2018 | 29/06/2022 | Martin Chak |
| Description | A commonly used paradigm for sampling and optimisation methods is the Langevin dynamics. There has been evidence that the second order methods that use momentum are more effective in exploring the state space than first order methods and in a certain case in optimisation, there is clear theoretical proof that the inclusion of momentum improves the rate of convergence. This award has allowed research that explores the use of higher order dynamics, in particular, of the generalised Langevin dynamics, that introduce an auxiliary variable and is equivalent to incorporating memory of past information over the trajectory of the dynamics into the time evolution. This has given promising results that show performance at the level of and above the second order methods in the case of nonconvex optimisation. |
| Exploitation Route | Higher order dynamics may be used as a gradient-based tool in high dimensional sampling and optimisation problems. These include the computation of numerical approximations to expectations with respect to probability measures such as the macroscopic quantities considered in molecular dynamics and finding optimal parameters to a given objective function such as those considered in machine learning. |
| Sectors | Digital/Communication/Information Technologies (including Software) Financial Services and Management Consultancy Manufacturing including Industrial Biotechology |
| URL | https://arxiv.org/abs/2003.06448 |