Attractors and computational properties of input-driven recurrent neural networks

Many of the mathematical techniques for understanding dynamical systems are restricted to input-free (autonomous) systems. However, for many applications, understanding the response when driven by an input is vital - this means we need to understand the behaviour of nonautonomous dynamical system. Neural networks are increasingly prevalent but despite their ubiquity, they often operate as "black boxes" that are trained according to heuristic algorithms and little is known about their internal function once trained. This is especially the case for recurrent neural networks (RNNs) which have internal dynamical states. For example, little is known about when a trained RNN will malfunction on given an input where it might be expected to function correctly.

This PhD project will approach these problem by examining the behaviour of recurrent neural networks with input, using tools such as pullback attractors from nonautonomous dynamical systems. The project will build on recent work (DOI:10.1016/j.physd.2020.132609) of the supervisor and collaborators about the relationship between pullback attractors, multistability of dynamical systems and computational properties of RNNs. The project will aim to characterize the responses of driven nonlinear dynamical systems in general (and RNNs in particular) and how they depend on inputs. This promises to give insights to repeatability, as well as function and malfunction of RNNs and their limits as computational devices. The project will develop a mathematical framework that can be applied to examples of gated neural networks where there is adaptation not only of connection weights but also of parameters that set timescales within the network.