Evaluation of costs and benefits of actions in the basal ganglia

We propose to investigate how the brain evaluates the costs and benefits of available options during decision making. We focus on a part of the brain called the basal ganglia that plays an important role in action selection, or a decision about which movement to take. This part of the brain is also involved in the evaluation of options during more abstract decisions. In the basal ganglia there are two main groups of neurons: One group has been shown to facilitate the choice or movement, and we refer to them as the Go neurons, while the other group has been shown to block or prevent movements, and we refer to them as the No-Go neurons. The activity of these two types of neurons is modulated by another population of neurons which release a chemical substance called dopamine. The presence of dopamine increases the activity of Go neurons and decreases the activity of No-Go neurons.

Despite years of studies, it is still highly debated what the functions of these two groups of neurons are. One dominant hypothesis is that the while Go neurons facilitate an action, the No-Go neurons block alternative choices, to ensure that only one action is chosen at the time. Alternatively, it has been recently proposed that the Go and No-Go neurons encode the payoffs and costs of actions, while the dopaminergic neurons encode the current motivational state, e.g. hunger. This theory suggests that the basal ganglia circuit weights the payoffs and costs differently according to the motivational state. For example, when an animal is hungry, a high level of dopamine increases the activity of Go neurons and decreases the activity of No-Go neurons, so that the payoffs of actions are weighted more than their costs. However these two hypotheses have not been yet directly tested in experimental data. This proposal aims at answering the fundamental questions concerning the Go and No-Go neurons: what information do they represent, how this information is integrated during choice and modulated by the activity of neurons releasing dopamine, and how the Go and No-Go neurons learn.

We propose to record the activity of Go and No-Go and neurons releasing dopamine while mice make choices between two levers associated with different amounts of reward and effort required to obtain it. In most past experiments studying the activity of the Go and No-Go neurons, their activity was recorded via electrodes inserted into animals' brains, but as the Go and No-Go neurons are intermixed in the brain, it is difficult to identify which of them generate the electrical activity. Therefore, in our study we will use a special technique allowing to record the activity of just one group of neurons. To record the Go neurons we will use special genetically modified mice, in which the Go neurons emit light whenever they produce activity. In these mice only the Go neurons emit light, so the light uniquely signals the activity of Go (rather than No-Go) neurons. To measure this light, two optic fibres will be inserted to two sides of the brain. Two additional groups of animals will be used that produce light during the activity of No-Go and neurons releasing dopamine respectively.

We will analyse the activity of Go and No-Go neurons during decision making, and investigate if and how they separately encode payoffs and costs of the options chosen by the animal. We will also study how the activity of neurons releasing dopamine influences decision making. Furthermore, by inspecting the changes in the activity over the experiment we will investigate how the Go and No-Go neurons learn about payoffs and costs of actions. Answering these questions is important because action selection and learning in the basal ganglia are affected in Parkinson's disease and several other disorders. Thus understanding how this system operates in the healthy brain is crucial for development of effective treatments that aim at restoring normal function.

We propose to investigate how the neural circuits in the basal ganglia evaluate costs and benefits of options. The basal ganglia is organized in two main pathways related to the initiation and inhibition of movements. These two pathways originate from separate populations of medium spiny neurons in striatum which express different dopaminergic receptors, to which we refer as D1 and D2-MSNs. Dopamine changes the balance between the two pathways as it increases excitability of the D1-MSNs and reduces the excitability of the D2-MSNs. However, the function of these two pathways is still hotly debated. A popular hypothesis postulates that while D1-MSNs encode evidence for selecting an action, D2-MSNs block alternative motor plans. On the other hand it has been suggested that D1 and D2-MSNs separately encode the payoffs and costs of actions, and the dopaminergic neurons encode the motivational state that can bias how the payoffs and costs are weighted during choice. The proposed research will use a combination of experimental and theoretic techniques to answer the fundamental questions on how D1 and D2-MSNs represent, use and learn the information on the value of actions.

We will record the population activity of D1, D2-MSNs and dopaminergic neurons of mice making choices between two options differing in payoff and effort required for reward delivery. We will employ photometry to measure neural activity in three groups of genetically modified animals in which light is emitted during the activity of D1, D2-MSNs and dopaminergic neurons. These data will allow us to test the main predictions of the hypotheses concerning the function of D1 and D2-MSNs. We will also employ a mathematical model of decision making to identify how the information encoded by the studied neural population is combined during choice. Furthermore, we will compare how well different computational models of learning can account for the changes in activity of D1, D2-MSNs and dopaminergic neurons.

The proposed research will directly benefit the academics in several disciplines (as described in the previous section), and long term the applications of the obtained results can improve health, economy and wellbeing of people in UK and beyond. In this section we focus on the latter benefits. Here we summarize who can benefit from the proposed research and in what way, while we provide the details of our plans to facilitate transfer of generated knowledge in the "Pathways to impact" document.

The first group who can in a long term benefit from the proposed research are the patients suffering from the disorders affecting decision networks in the basal ganglia such as Parkinson's disease and depression. The current treatments for these disorders are not fully ameliorating the symptoms, have side effects, and are often not personalized to the specific disruptions in patients' brain. The reason for this, is that many of these treatments were developed by trial and error, rather than being based on the understanding of the affected mechanisms. For example deep brain stimulation involves implanting electrodes into patients' brain and stimulating with a high frequency current. Although this treatment is routinely used for Parkinson's disease and has been used also for depression, the effects of the stimulation on underlying computation in the brain are not understood. This project will provide a precise description of the operation of the brain circuits which are malfunctioning in these disorders. This will open a possibility of a new approach to developing treatments that aims at restoring the computations performed by the circuits in the healthy brain. Such treatments are likely to better ameliorate symptoms, have fewer side-effects and be personalized to individual patients.

Second, the proposed research will benefit Pharmaceutical and Biotechnology industry, which develops treatments for the disorders of the basal ganglia system. As described above, the insights provided by the research will allow work on more refined treatments.

The proposed research can also long term benefit people who take decisions having effects on economy, and important choices in life, i.e. all of us. The effects of motivational state on the integration of costs and benefits, studied in this proposal, have significant impact on important decisions. For example, Dazinger et al. (2011) reported that parole decisions made by experienced judges are to a large extent determined by how long before they had their last meal. The description of the mechanisms of cost-benefit decision making in the brain could long term lead to the development of training protocols which would make people aware of how their current state (hunger, mood, etc.) affects the levels of neuromodulators in the brain, which in turn changes the weighting of payoffs and costs. Taking these biases into account can facilitate making more informed and better decisions. Therefore, the results of the proposed research are of great interest for people making decision making in various fields, and to general public.

Finally, the proposal will bring benefits to British economy through training and education that will occur as a part of this project. Two postdoctoral researchers will receive interdisciplinary training which will combine biological and computational skills. Since integrating these areas can generate much synergy, training people that who can in the future lead work on the interface of these disciplines is of great benefit to both academia and biotechnology industry. Furthermore, the planned public engagement activities (see Pathways to impact) will encourage high school pupils to study STEM subjects, which will again in a long term strengthen British economy.

Reference
Danziger, S., Levav, J., & Avnaim-Pesso, L. (2011). Extraneous factors in judicial decisions. Proceedings of the National Academy of Sciences, 108, 6889-6892.