Neural networks with frugal learning

The amount of metabolic energy available to the brain is limited. Hence energy efficiency is believed to have shaped many aspects of the brain's design and how it performs computations. In addition to the energy required by the brain to compute, changing the brain's connections, which happens during learning, is believed to require large amounts of energy as well. The computational consequences of this cost on learning and memory are not known.
This project will analyse its impact on learning, by comparing the energy usage of different models of learning in neural networks, and testing different synaptic learning rules.
Energy efficiency is an area of research with high potential in both computational neuroscience and machine learning. This project will further explore a normative view of the brain, emphasizing a principle of energy efficiency, in regards to the synaptic updates involved in learning. The main question it will attempt to address will be how biological neural networks are able to perform learning despite strong energy constraints. Additionally, it may address how this information could help the development of more efficient, biologically-inspired machine learning models. The expected outcomes of the project are therefore that it will lead to a more accurate understanding of how learning is implemented in biological neural systems, and that it will lead to better artificial neural systems. The project's scope, analysing how energy-efficient biological networks perform learning, is novel, and it builds upon recent research on the energy impact of learning.
The methods will be analytical calculations and computational simulations of learning in artificial neural networks.
The project falls in the following EPSRC research areas "Artificial intelligence technologies", "Biological informatics" and "Microelectronic device technology".
We have an informal collaboration with the lab of Prof Preat in Paris who performs experiments on the energy use associated to learning in fruit flies. We are also seeking collaboration with Google research.