Understanding Neural Networks through Dynamics

Artificial Neural Networks (ANNs) have recently moved back into the spotlight as tools to simulate, predict, and classify complex inputs. Our ability to use these new computational methods with confidence in applications demands that we develop a better fundamental understanding of why, how, and in what situations ANNs can be relied on. Reliance on such tools for decision support is increasing rapidly in fields from autonomous vehicles to policing, as is the list of notable examples of biases, failures, and injustices that can all be traced back to fundamental mathematical issues.

There has been particular focus recently on reservoir computing, related to recurrent neural networks, in which most of the network connections form a single-layer recurrent 'dynamical reservoir' with fixed (usually randomly chosen) connections and training carried out by adjusting a small number of output weights. In this project the work will focus on a class of reservoir systems known as Echo State Networks (ESNs).

This proposal will develop new fundamental understanding of structure in ESNs, and will develop previously-unexploited properties of ESNs as dynamical systems; in turn this will enable new applications of ESNs to physical systems, including climate dynamics.