MLTURB: A new understanding of turbulence via a machine-learnt dynamical systems theory
Lead Research Organisation:
University of Edinburgh
Department Name: Sch of Mathematics
Abstract
Turbulence at very large scales is a complex, nonlinear problem of fundamental scientific and
societal importance. Turbulent computations often rely on a "subgrid" model for the smaller,
dissipative scales of motion. However, accurate simulation and prediction in the large-scale flows
of paramount industrial and geophysical importance requires a subgrid which covers a greater
range of scales - and hence must encode more of the turbulent dynamics. There is, therefore, a
pressing need for answers to long-standing questions on the dynamics of energy transfer
mechanisms in strongly turbulent fluids. This proposal is focused on establishing this new
understanding, focusing on both the dynamical processes at play in the turbulent energy cascade
and the rare, small-scale intermittent bursting events which are associated with extreme local
values of drag or heat transfer. The new methodology is rooted in dynamical systems theory built
around exact, unstable solutions of the governing equations. This approach has been
transformative in transitional/weakly turbulent flows, but has so far proved challenging to apply in
parameter regimes of industrial relevance, due to the difficulties associated with identifying and
converging the unstable solutions. These limitations are overcome here via a new approach using
novel machine learning algorithms and differentiable programming - with the necessary compute
power and expertise provided through a collaboration with Google's Accelerated Sciences team.
These tools are complemented by a robust low-order modelling framework based on the Koopman
operator, which will be used both as both a tool to understand the dynamics encapsulated in the
unstable solutions (and around them in phase space) and also to probe even more strongly
turbulent flows to establish the dominant mechanisms and assess exactly which dynamical
processes are required in the subgrid scale models of the future.
societal importance. Turbulent computations often rely on a "subgrid" model for the smaller,
dissipative scales of motion. However, accurate simulation and prediction in the large-scale flows
of paramount industrial and geophysical importance requires a subgrid which covers a greater
range of scales - and hence must encode more of the turbulent dynamics. There is, therefore, a
pressing need for answers to long-standing questions on the dynamics of energy transfer
mechanisms in strongly turbulent fluids. This proposal is focused on establishing this new
understanding, focusing on both the dynamical processes at play in the turbulent energy cascade
and the rare, small-scale intermittent bursting events which are associated with extreme local
values of drag or heat transfer. The new methodology is rooted in dynamical systems theory built
around exact, unstable solutions of the governing equations. This approach has been
transformative in transitional/weakly turbulent flows, but has so far proved challenging to apply in
parameter regimes of industrial relevance, due to the difficulties associated with identifying and
converging the unstable solutions. These limitations are overcome here via a new approach using
novel machine learning algorithms and differentiable programming - with the necessary compute
power and expertise provided through a collaboration with Google's Accelerated Sciences team.
These tools are complemented by a robust low-order modelling framework based on the Koopman
operator, which will be used both as both a tool to understand the dynamics encapsulated in the
unstable solutions (and around them in phase space) and also to probe even more strongly
turbulent flows to establish the dominant mechanisms and assess exactly which dynamical
processes are required in the subgrid scale models of the future.
Publications
Cleary A
(2023)
Exploring the free-energy landscape of a rotating superfluid.
in Chaos (Woodbury, N.Y.)
Page J
(2024)
Exact coherent structures in two-dimensional turbulence identified with convolutional autoencoders
in Journal of Fluid Mechanics
Page J
(2024)
Recurrent flow patterns as a basis for two-dimensional turbulence: Predicting statistics from structures.
in Proceedings of the National Academy of Sciences of the United States of America
Page J
(2025)
Super-resolution of turbulence with dynamics in the loss
in Journal of Fluid Mechanics
| Title | Code for periodic orbit computation in turbulence |
| Description | This GitHub repository contains code to search for exact solutions of the Navier-Stokes equations using gradient-based optimisation with a fully differentiable flow solver. There are several novel features: 1. Written in the JAX library for use with GPUs and combination with neural network traiining 2. Ability to pre-compile time marching routines with a target time that changes over the optimisation loop 3. Removes the need for previous restrictions in the search for exact solutions (e.g. near recurrence) leading to orders of magnitude improvement in number of convergences |
| Type Of Material | Computer model/algorithm |
| Year Produced | 2024 |
| Provided To Others? | Yes |
| Impact | Preprint: A. Cleary & J. Page, Dynamical relevance of periodic orbits under increasing Reynolds number and connections to inviscid dynamics [arXiv:2502.06475] Papers relying on this methodology: J. Page, Super-resolution of turbulence with dynamics in the loss, Journal of Fluid Mechanics 1002, R3 (2025) J. Page, P. Norgaard, M. P. Brenner, R. R. Kerswell, Recurrent flow patterns as a basis for two-dimensional turbulence: predicting statistics from structures, Proceedings of the National Academy of Sciences 121, 23 (2024) A. Cleary & J. Page, Exploring the free-energy landscape of a rotating superfluid, Chaos 33, 103123 (2023) |
| URL | https://github.com/JacobSRPage/upo-ad |
| Title | Code for super-resolution of turbulence with dynamical loss functions |
| Description | This is a GitHub repository containing code for training neural networks to perform "super resolution" of under-resolved turbulent snapshots. Once trained, these models take a coarse grained snapshot of a turbulent flow and produce a high-resolution prediction. This code is written in JAX to allow for coupling of the network training with a (differentiable) flow solver. This means that predictions can be unrolled in time when the loss is calculated, which allows networks to be trained in an approach similar to data-assimilation. The key advance is that this means accurate models can be trained without access to high resolution reference data, as might be expected in an experiment. |
| Type Of Material | Computer model/algorithm |
| Year Produced | 2025 |
| Provided To Others? | Yes |
| Impact | Paper: J. Page, Super-resolution of turbulence with dynamics in the loss, Journal of Fluid Mechanics 1002, R3 (2025) Ongoing collaboration with Rigas group (Imperial College London) -- see collaboration section |
| URL | https://github.com/JacobSRPage/super-res-dynamical |
| Title | Neural networks weights related to "Recurrent patterns as a basis for two-dimensional turbulence: predicting statistics from structures" |
| Description | The dataset contains neural network weights (checkpointed in TensorFlow) for two deep-convolutional autoencoders designed to generate low-dimensional representations of snapshots of vorticity in two-dimensional turbulence. The models have the same architecture apart from the size of the inner-most "embedding" layer. Code to construct the model architecture is also included as a python script. For details of loss function and training protocol please see associated publication "Recurrent patterns as a basis for two-dimensional turbulence: predicting statistics from structures" (accepted in PNAS, 2024) |
| Type Of Material | Database/Collection of data |
| Year Produced | 2024 |
| Provided To Others? | Yes |
| Impact | These neural networks allow one to accurately label a turbulent trajectory according to which exact unstable periodic solution it is "closest to" in state space. These ideas can be used (and were used by my group) to convert a turbulent simulation to a Markov chain between unstable states. The invariant measure of the chain can be used to predict full statistics of the turbulence in terms of the statistics of the exact solutions, providing a tantalising connection between dynamical events and statistics in turbulence, which is a long-held objective in this field. Associated publication: "Recurrent flow patterns as a basis for two-dimensional turbulence: predicting statistics from structures", Page et al., Proceedings of the National Academy of Sciences 121 (2024) |
| URL | https://datashare.ed.ac.uk/handle/10283/8779 |
| Title | Neural networks weights related to "Super-resolution of turbulence with dynamics in the loss" |
| Description | A set of routines to train neural networks to perform super-resolution, without necessarily requiring high-resolution data, along with network weights used to generate figures in the publication "Super-resolution of turbulence with dynamics in the loss", (J. Page, Journal of Fluid Mechanics, accepted 2024). Implementation wraps around the spectral version of JAX-CFD (https://github.com/google/jax-cfd). Neural networks are written in Keras using the JAX backend. This dataset consists of a series of python scripts and .h5 files of network weights, organised into subdirectories: code/ # scripts used to train and analyse the networks (python, requires installation of keras + JAX backend, JAX-CFD) paper_weights/ # weights for the neural networks as documented in the paper. # subdirectories here point to different Reynolds numbers (100, 1000) with weights at M = 16 and M = 32 coarse-graining # an additional subdirectory includes networks trained with noisy data Neural network weights are saved as .h5 files. An example script is included ("load_weights.py") to illustrate how to load into a model. |
| Type Of Material | Database/Collection of data |
| Year Produced | 2024 |
| Provided To Others? | Yes |
| Impact | Paper: J. Page, Super-resolution of turbulence with dynamics in the loss, Journal of Fluid Mechanics 1002, R3 (2025) |
| URL | https://datashare.ed.ac.uk/handle/10283/8921 |
| Description | Edinburgh/Cambridge/Harvard/Google - ML for turbulence |
| Organisation | |
| Department | Research at Google |
| Country | United States |
| Sector | Private |
| PI Contribution | Leading the theoretical and computational development of ML-based methods for searching for exact coherent structures in turbulence. |
| Collaborator Contribution | Weekly meetings -- scientific discussions of research ideas, directions etc. Co-writing papers. Compute time (on GPUs) provided by both Google and Harvard for model training. |
| Impact | J. Page, J. Holey, M. P. Brenner, R. R. Kerswell, Exact coherent structures in two-dimensional turbulence identified with convolutional autoencoders, Journal of Fluid Mechanics 991, A10 (2024) J. Page, P. Norgaard, M. P. Brenner, R. R. Kerswell, Recurrent flow patterns as a basis for two-dimensional turbulence: predicting statistics from structures, Proceedings of the National Academy of Sciences 121, 23 (2024) J. Page, M. P. Brenner & R. R. Kerswell, Revealing the state space of turbulence using machine learning, Physical Review Fluids 6, 034402 (2021) [pre award] |
| Start Year | 2018 |
| Description | Edinburgh/Cambridge/Harvard/Google - ML for turbulence |
| Organisation | Harvard University |
| Country | United States |
| Sector | Academic/University |
| PI Contribution | Leading the theoretical and computational development of ML-based methods for searching for exact coherent structures in turbulence. |
| Collaborator Contribution | Weekly meetings -- scientific discussions of research ideas, directions etc. Co-writing papers. Compute time (on GPUs) provided by both Google and Harvard for model training. |
| Impact | J. Page, J. Holey, M. P. Brenner, R. R. Kerswell, Exact coherent structures in two-dimensional turbulence identified with convolutional autoencoders, Journal of Fluid Mechanics 991, A10 (2024) J. Page, P. Norgaard, M. P. Brenner, R. R. Kerswell, Recurrent flow patterns as a basis for two-dimensional turbulence: predicting statistics from structures, Proceedings of the National Academy of Sciences 121, 23 (2024) J. Page, M. P. Brenner & R. R. Kerswell, Revealing the state space of turbulence using machine learning, Physical Review Fluids 6, 034402 (2021) [pre award] |
| Start Year | 2018 |
| Description | Edinburgh/Cambridge/Harvard/Google - ML for turbulence |
| Organisation | University of Cambridge |
| Department | Department of Applied Mathematics and Theoretical Physics (DAMTP) |
| Country | United Kingdom |
| Sector | Academic/University |
| PI Contribution | Leading the theoretical and computational development of ML-based methods for searching for exact coherent structures in turbulence. |
| Collaborator Contribution | Weekly meetings -- scientific discussions of research ideas, directions etc. Co-writing papers. Compute time (on GPUs) provided by both Google and Harvard for model training. |
| Impact | J. Page, J. Holey, M. P. Brenner, R. R. Kerswell, Exact coherent structures in two-dimensional turbulence identified with convolutional autoencoders, Journal of Fluid Mechanics 991, A10 (2024) J. Page, P. Norgaard, M. P. Brenner, R. R. Kerswell, Recurrent flow patterns as a basis for two-dimensional turbulence: predicting statistics from structures, Proceedings of the National Academy of Sciences 121, 23 (2024) J. Page, M. P. Brenner & R. R. Kerswell, Revealing the state space of turbulence using machine learning, Physical Review Fluids 6, 034402 (2021) [pre award] |
| Start Year | 2018 |
| Description | Page/Rigas RL collaboration |
| Organisation | Imperial College London |
| Country | United Kingdom |
| Sector | Academic/University |
| PI Contribution | One of my PhD students is being co-supervised by Dr Georgios Rigas (Aeronautics at Imperial College London). In addition to the student, we contribute expertise in dynamical systems theory, supervised learning and turbulence, along with compute time. |
| Collaborator Contribution | Co-supervision of the student (weekly meetings and hosting for extended stays -- 2 x 2 week-long visits so far) and expertise in reinforcement learning and control theory. |
| Impact | None yet. |
| Start Year | 2024 |
