Compositional Learning algorithms

Neural networks have become an increasingly popular tool for solving many real-world problems. They are a general framework for differentiable optimization which includes many other machine learning approaches as special cases. There are two interesting properties of neural networks related to compositionality: (i) they are compositional - increasing the number of layers tends to yield better performance, and (ii) they are discovering (compositional) structures in data.
The main goal of this thesis is to use the language of category theory to describe and quantify compositionality of neural networks. Concretely, the focus is to explore and frame complex interactions between networks, i.e. learning agents in a compositional way.

This includes (i) expansion of the existing work on compositional supervised learning into territories of unsupervised learning, generative models and the setting of multiple interacting agents, (ii) exploration of how categorical structures themselves can be learned using gradient descent, and (iii) precise specification the software implementation of these networks in a pure functional language.

Compositionality describes and quantifies how complex things can be assembled out of simpler parts. In order to build systems that scale, orders of magnitude larger than contemporary software, one needs to take a radical approach to complexity management and control of emergent behaviour.

In this research proposal, we outline the idea of using the language of category theory as the main tool for design of such compositional systems and building bridges between disparate fields.