Adjoint sensitivity methods
Lead Research Organisation:
University of Oxford
Abstract
The derivative of a function measures the sensitivity of the target output variable to changes in the input parameters. Several applications require the calculation of derivatives with respect to a large number of input parameters. The financial market, for example, is driven by many factors (e.g. interest rates) and it is key to understand how changes in each factor can affect the price of a financial product.
If we individually compute derivatives for each input parameter, computational cost scales linearly with the number of input parameters. Linear cost is often not good enough for a large/vast amount number of input parameters. Instead, using an adjoint formulation yields a surprising result: "Given a computer program to compute a scalar output, we can obtain the sensitivity of that output to all the program's inputs for a cost which is no more than a factor 4 greater than the original evaluation cost." (Preprint of the book "Smoking Adjoints" by Prof. Mike Giles). These adjoint sensitivity methods go under the name of automatic (or algorithmic) differentiation in computer science, backpropagation in machine learning, and AAD in finance.
In finance, asset price dynamics are often assumed to follow a (random) model. A necessary requirement of a good model is that it is able to match prices observed in the market as accurately as possible. In practice, a financial model often has free parameters that are calibrated in such a way that minimizes the difference between prices resulting from the model and prices observed in the market. In order to find the best choice for the free parameters, one might proceed iteratively with the search direction given by the sensitivity of the price difference to changes in the free parameters (gradient-based optimization approach). Thus, one needs to compute derivatives for each iteration. While adjoint sensitivity methods guarantee the efficient computation of derivatives of one financial product, each financial product requires a new sensitivity computation. As a consequence, computational cost increases as the number of financial products increases. This research project develops a technique to render computational cost independent of the number of financial products used in the calibration procedure.
By decreasing degree of relevance, this project falls within the following EPSRC research areas: (1) Numerical Analysis, (2) Operational Research, (3) Artificial Intelligence Technologies. The research project is in collaboration with HSBC. The aim of our collaboration is to compute adjoint sensitivities of a financial workflow with multiple components that are approximated by neural networks.
If we individually compute derivatives for each input parameter, computational cost scales linearly with the number of input parameters. Linear cost is often not good enough for a large/vast amount number of input parameters. Instead, using an adjoint formulation yields a surprising result: "Given a computer program to compute a scalar output, we can obtain the sensitivity of that output to all the program's inputs for a cost which is no more than a factor 4 greater than the original evaluation cost." (Preprint of the book "Smoking Adjoints" by Prof. Mike Giles). These adjoint sensitivity methods go under the name of automatic (or algorithmic) differentiation in computer science, backpropagation in machine learning, and AAD in finance.
In finance, asset price dynamics are often assumed to follow a (random) model. A necessary requirement of a good model is that it is able to match prices observed in the market as accurately as possible. In practice, a financial model often has free parameters that are calibrated in such a way that minimizes the difference between prices resulting from the model and prices observed in the market. In order to find the best choice for the free parameters, one might proceed iteratively with the search direction given by the sensitivity of the price difference to changes in the free parameters (gradient-based optimization approach). Thus, one needs to compute derivatives for each iteration. While adjoint sensitivity methods guarantee the efficient computation of derivatives of one financial product, each financial product requires a new sensitivity computation. As a consequence, computational cost increases as the number of financial products increases. This research project develops a technique to render computational cost independent of the number of financial products used in the calibration procedure.
By decreasing degree of relevance, this project falls within the following EPSRC research areas: (1) Numerical Analysis, (2) Operational Research, (3) Artificial Intelligence Technologies. The research project is in collaboration with HSBC. The aim of our collaboration is to compute adjoint sensitivities of a financial workflow with multiple components that are approximated by neural networks.
Planned Impact
Probabilistic modelling permeates the Financial services, healthcare, technology and other Service industries crucial to the UK's continuing social and economic prosperity, which are major users of stochastic algorithms for data analysis, simulation, systems design and optimisation. There is a major and growing skills shortage of experts in this area, and the success of the UK in addressing this shortage in cross-disciplinary research and industry expertise in computing, analytics and finance will directly impact the international competitiveness of UK companies and the quality of services delivered by government institutions.
By training highly skilled experts equipped to build, analyse and deploy probabilistic models, the CDT in Mathematics of Random Systems will contribute to
- sharpening the UK's research lead in this area and
- meeting the needs of industry across the technology, finance, government and healthcare sectors
MATHEMATICS, THEORETICAL PHYSICS and MATHEMATICAL BIOLOGY
The explosion of novel research areas in stochastic analysis requires the training of young researchers capable of facing the new scientific challenges and maintaining the UK's lead in this area. The partners are at the forefront of many recent developments and ideally positioned to successfully train the next generation of UK scientists for tackling these exciting challenges.
The theory of regularity structures, pioneered by Hairer (Imperial), has generated a ground-breaking approach to singular stochastic partial differential equations (SPDEs) and opened the way to solve longstanding problems in physics of random interface growth and quantum field theory, spearheaded by Hairer's group at Imperial. The theory of rough paths, initiated by TJ Lyons (Oxford), is undergoing a renewal spurred by applications in Data Science and systems control, led by the Oxford group in conjunction with Cass (Imperial). Pathwise methods and infinite dimensional methods in stochastic analysis with applications to robust modelling in finance and control have been developed by both groups.
Applications of probabilistic modelling in population genetics, mathematical ecology and precision healthcare, are active areas in which our groups have recognized expertise.
FINANCIAL SERVICES and GOVERNMENT
The large-scale computerisation of financial markets and retail finance and the advent of massive financial data sets are radically changing the landscape of financial services, requiring new profiles of experts with strong analytical and computing skills as well as familiarity with Big Data analysis and data-driven modelling, not matched by current MSc and PhD programs. Financial regulators (Bank of England, FCA, ECB) are investing in analytics and modelling to face this challenge. We will develop a novel training and research agenda adapted to these needs by leveraging the considerable expertise of our teams in quantitative modelling in finance and our extensive experience in partnerships with the financial institutions and regulators.
DATA SCIENCE:
Probabilistic algorithms, such as Stochastic gradient descent and Monte Carlo Tree Search, underlie the impressive achievements of Deep Learning methods. Stochastic control provides the theoretical framework for understanding and designing Reinforcement Learning algorithms. Deeper understanding of these algorithms can pave the way to designing improved algorithms with higher predictability and 'explainable' results, crucial for applications.
We will train experts who can blend a deeper understanding of algorithms with knowledge of the application at hand to go beyond pure data analysis and develop data-driven models and decision aid tools
There is a high demand for such expertise in technology, healthcare and finance sectors and great enthusiasm from our industry partners. Knowledge transfer will be enhanced through internships, co-funded studentships and paths to entrepreneurs
By training highly skilled experts equipped to build, analyse and deploy probabilistic models, the CDT in Mathematics of Random Systems will contribute to
- sharpening the UK's research lead in this area and
- meeting the needs of industry across the technology, finance, government and healthcare sectors
MATHEMATICS, THEORETICAL PHYSICS and MATHEMATICAL BIOLOGY
The explosion of novel research areas in stochastic analysis requires the training of young researchers capable of facing the new scientific challenges and maintaining the UK's lead in this area. The partners are at the forefront of many recent developments and ideally positioned to successfully train the next generation of UK scientists for tackling these exciting challenges.
The theory of regularity structures, pioneered by Hairer (Imperial), has generated a ground-breaking approach to singular stochastic partial differential equations (SPDEs) and opened the way to solve longstanding problems in physics of random interface growth and quantum field theory, spearheaded by Hairer's group at Imperial. The theory of rough paths, initiated by TJ Lyons (Oxford), is undergoing a renewal spurred by applications in Data Science and systems control, led by the Oxford group in conjunction with Cass (Imperial). Pathwise methods and infinite dimensional methods in stochastic analysis with applications to robust modelling in finance and control have been developed by both groups.
Applications of probabilistic modelling in population genetics, mathematical ecology and precision healthcare, are active areas in which our groups have recognized expertise.
FINANCIAL SERVICES and GOVERNMENT
The large-scale computerisation of financial markets and retail finance and the advent of massive financial data sets are radically changing the landscape of financial services, requiring new profiles of experts with strong analytical and computing skills as well as familiarity with Big Data analysis and data-driven modelling, not matched by current MSc and PhD programs. Financial regulators (Bank of England, FCA, ECB) are investing in analytics and modelling to face this challenge. We will develop a novel training and research agenda adapted to these needs by leveraging the considerable expertise of our teams in quantitative modelling in finance and our extensive experience in partnerships with the financial institutions and regulators.
DATA SCIENCE:
Probabilistic algorithms, such as Stochastic gradient descent and Monte Carlo Tree Search, underlie the impressive achievements of Deep Learning methods. Stochastic control provides the theoretical framework for understanding and designing Reinforcement Learning algorithms. Deeper understanding of these algorithms can pave the way to designing improved algorithms with higher predictability and 'explainable' results, crucial for applications.
We will train experts who can blend a deeper understanding of algorithms with knowledge of the application at hand to go beyond pure data analysis and develop data-driven models and decision aid tools
There is a high demand for such expertise in technology, healthcare and finance sectors and great enthusiasm from our industry partners. Knowledge transfer will be enhanced through internships, co-funded studentships and paths to entrepreneurs
Studentship Projects
| Project Reference | Relationship | Related To | Start | End | Student Name |
|---|---|---|---|---|---|
| EP/S023925/1 | 01/04/2019 | 30/09/2027 | |||
| 2435699 | Studentship | EP/S023925/1 | 01/10/2020 | 30/09/2024 | Filippo De Angelis |