Attractors and computational properties of input-driven recurrent neural networks

Lead Research Organisation: University of Exeter
Department Name: Mathematics

Abstract

Many of the mathematical techniques for understanding dynamical systems are restricted to input-free (autonomous) systems. However, for many applications, understanding the response when driven by an input is vital - this means we need to understand the behaviour of nonautonomous dynamical system. Neural networks are increasingly prevalent but despite their ubiquity, they often operate as "black boxes" that are trained according to heuristic algorithms and little is known about their internal function once trained. This is especially the case for recurrent neural networks (RNNs) which have internal dynamical states. For example, little is known about when a trained RNN will malfunction on given an input where it might be expected to function correctly.

This PhD project will approach these problem by examining the behaviour of recurrent neural networks with input, using tools such as pullback attractors from nonautonomous dynamical systems. The project will build on recent work (DOI:10.1016/j.physd.2020.132609) of the supervisor and collaborators about the relationship between pullback attractors, multistability of dynamical systems and computational properties of RNNs. The project will aim to characterize the responses of driven nonlinear dynamical systems in general (and RNNs in particular) and how they depend on inputs. This promises to give insights to repeatability, as well as function and malfunction of RNNs and their limits as computational devices. The project will develop a mathematical framework that can be applied to examples of gated neural networks where there is adaptation not only of connection weights but also of parameters that set timescales within the network.

Publications

10 25 50

Studentship Projects

Project Reference Relationship Related To Start End Student Name
EP/W523859/1 30/09/2021 29/09/2025
2606311 Studentship EP/W523859/1 30/09/2021 29/09/2025 Muhammed Fadera