Deep temporal credit assignment with dynamic synapses

(as written by supervisor Rui Ponte Costa/not final)

Successful behaviour relies on the ability of animals and machines to predict the outcome of their actions. Selecting which synapses (or parameters) to adjust to trigger the right action is a hard problem as it depends on past sensory information. This is one of the most fundamental features of the brain, but exactly how it solves such temporal credit assignment problems has remained largely enigmatic. Inspired by machine learning optimisation methods that underlie the recent deep learning revolution we have recently introduced a novel theoretical optimization framework. This framework allows us to derived a recurrent neural network with multiple layers, neuron types and synaptic learning rules akin to the ones found in the brain [1] (Costa et al. in prep.). One of the most striking observations of this analysis is the need for a synaptic eligibility trace, which tracks incoming activity from presynaptic neurons. Growing evidence suggests that such eligibility traces might be encoded by internal synaptic dynamics [1-5]. In this project, in close interaction with the lab of Jack Mellor, the student will start out by extending the existing theoretical optimization framework to consider the role of synaptic dynamics and test it in simple tasks (Aim 1 ). The resulting model will be compared with the wide range of experimental neuroscience data available at Jack Mellor's and Matt Jones's labs using probabilistic inference in collaboration with Carl Henrik Ek (Aim 2 ). Finally, this biologically plausible model will be tested in more challenging tasks in collaboration with Dima Almaden and external collaborators (Aim 3 ). Overall, this project will provide a compelling proposal for how the brain encodes the information needed for temporal credit assignment, which may in turn inspire new machine learning algorithms.