Distributed anatomical circuits for decision-making, inference, and learning
Lead Research Organisation:
University of Oxford
Department Name: Experimental Psychology
Abstract
Every day we make decisions about what to do next. We do this because we are constantly monitoring and evaluating how well things are going. As a consequence we adjust our behaviour so it is appropriate for the current context or we decide to take one course of action rather than another. Sometimes we are aware of making these evaluations and decisions but often we are not. Nevertheless, however much we take these abilities for granted, it is striking when they are altered in psychological illnesses such as depression.
The aim of this proposal is to undertake work to understand what the brain does to enable us to behave in the way that we do. Our focus is on parts of the brain called the prefrontal and cingulate cortex. We already know that these brain regions are especially important for the behaviour we are interested in but what we do not know is how they accomplish the role they play. We want to find out the mechanisms by which they operate and the way in which they interact with the rest of the brain.
A key part of the work is designing behavioural tasks to probe cognitive operations in a precise way to reveal their mechanistic basis. A second component is recording brain activity and seeing how it relates to behaviour. We do this by using a magnetic resonance imaging (MRI) scanner, usually by taking what are called functional MRI (fMRI) scans. FMRI scans tell us about blood oxygenation in the brain. This is useful because the blood oxygen level dependent (BOLD) signal tracks the activity of the brain's cells -- neurons -- in a very precise way. It is, for example, possible to estimate changes in distribution of BOLD signal in specific brain areas from moment to moment as a decision is made or as feedback is provided to enable adjustments and changes in behaviour.
We conduct the fMRI recording in animals because we also want to examine the consequences of manipulating the activity we record. This is essential for finding out what activity patterns are causally driving behaviours. We can test causation by making precise and circumscribed interventions in the brain. We do this under anaesthesia in the same way that it would be done with human patients. When the animals recover we monitor changes in behaviour. Usually there are no obvious changes in behaviour because the interventions we carry out are subtle. If, however, we have designed our behavioural tasks with care so as to precisely probe specific cognitive processes, then we may be able to pick up equally subtle alterations in behaviour. They can then be measured and quantified. We use macaques because they provide a model of many features of human prefrontal and cingulate cortex. Most other animals lack these features so they cannot be used as models.
One of the questions that we are examining concerns how quickly we should change and adjust our behaviour. There is evidence that in some psychological illnesses, such as bipolar depression, our behaviour becomes too responsive to each minor piece of feedback; the behaviour becomes too volatile. We are also interested in how, when we receive feedback for a choice, we attribute that feedback correctly to the event that really caused it. There is evidence that we do not always manage this simple job well and when we are very poor at it we may draw odd conclusions about what we are, and are not, responsible for. Again this may be a feature of psychological illnesses. A third process we are interested in is inference. Often neuroscientists have studied the neural mechanisms that mediate learning about particular events or actions. Once the learning is done a good decision can be made next time the event is encountered or the action is needed. However, often in the real world we make inferences about what to do next on the basis of experience of some situations with some similar component elements. We will attempt to understand how such inferences are made.
The aim of this proposal is to undertake work to understand what the brain does to enable us to behave in the way that we do. Our focus is on parts of the brain called the prefrontal and cingulate cortex. We already know that these brain regions are especially important for the behaviour we are interested in but what we do not know is how they accomplish the role they play. We want to find out the mechanisms by which they operate and the way in which they interact with the rest of the brain.
A key part of the work is designing behavioural tasks to probe cognitive operations in a precise way to reveal their mechanistic basis. A second component is recording brain activity and seeing how it relates to behaviour. We do this by using a magnetic resonance imaging (MRI) scanner, usually by taking what are called functional MRI (fMRI) scans. FMRI scans tell us about blood oxygenation in the brain. This is useful because the blood oxygen level dependent (BOLD) signal tracks the activity of the brain's cells -- neurons -- in a very precise way. It is, for example, possible to estimate changes in distribution of BOLD signal in specific brain areas from moment to moment as a decision is made or as feedback is provided to enable adjustments and changes in behaviour.
We conduct the fMRI recording in animals because we also want to examine the consequences of manipulating the activity we record. This is essential for finding out what activity patterns are causally driving behaviours. We can test causation by making precise and circumscribed interventions in the brain. We do this under anaesthesia in the same way that it would be done with human patients. When the animals recover we monitor changes in behaviour. Usually there are no obvious changes in behaviour because the interventions we carry out are subtle. If, however, we have designed our behavioural tasks with care so as to precisely probe specific cognitive processes, then we may be able to pick up equally subtle alterations in behaviour. They can then be measured and quantified. We use macaques because they provide a model of many features of human prefrontal and cingulate cortex. Most other animals lack these features so they cannot be used as models.
One of the questions that we are examining concerns how quickly we should change and adjust our behaviour. There is evidence that in some psychological illnesses, such as bipolar depression, our behaviour becomes too responsive to each minor piece of feedback; the behaviour becomes too volatile. We are also interested in how, when we receive feedback for a choice, we attribute that feedback correctly to the event that really caused it. There is evidence that we do not always manage this simple job well and when we are very poor at it we may draw odd conclusions about what we are, and are not, responsible for. Again this may be a feature of psychological illnesses. A third process we are interested in is inference. Often neuroscientists have studied the neural mechanisms that mediate learning about particular events or actions. Once the learning is done a good decision can be made next time the event is encountered or the action is needed. However, often in the real world we make inferences about what to do next on the basis of experience of some situations with some similar component elements. We will attempt to understand how such inferences are made.
Technical Summary
We will investigate the neural basis of decision making, learning, behavioural change, and inference in PFC and ACC. Neural activity will be recorded in macaques with fMRI and probed with a range of interventions including lesions, focal ultrasound neurostimulation, and possibly microstimulation. Effects on behaviour and neural activity will be measured.
The aim of each project is not simply to assess involvement of brain areas in cognitive processes of interest but instead it is to derive a mechanistic understanding of the way in which the areas operate and the way in which they interact with other brain areas. With careful task design it is possible to identify neural activity covarying with quantitative estimates of choice value and value comparison. It is then possible to determine the nature of PFC and ACC representations when a decision is made between choices, when an inference is made about a choice's value on the basis of past experience, and when estimates of choice values or behaviour values are being updated during learning.
Computational models will be used to guide analyses. The models will be used to make quantitative predictions of behaviour and neural activity that can be contrasted with actual observations.
We will investigate: 1) simultaneous representation of multiple learning rates in dACC; 2) different reward representations in lOFC and amygdala in relation to credit assignment; 3) independent versus interactive nature of vmPFC and dACC during decision making; 4) hippocampus-dependent map-like representation to guide decision making in vmPFC and pgACC; 5) model-based versus model-free reward learning mediated by lPFC and vmPFC interaction; 6) activation and suppression of stimulus-reward representations in OFC; 7) a mechanism for determining speed, accuracy, and task engagement dependent on average reward and temporal structure of the environment; 8) distinct representations of surprise and prediction error in PFC and striatum.
The aim of each project is not simply to assess involvement of brain areas in cognitive processes of interest but instead it is to derive a mechanistic understanding of the way in which the areas operate and the way in which they interact with other brain areas. With careful task design it is possible to identify neural activity covarying with quantitative estimates of choice value and value comparison. It is then possible to determine the nature of PFC and ACC representations when a decision is made between choices, when an inference is made about a choice's value on the basis of past experience, and when estimates of choice values or behaviour values are being updated during learning.
Computational models will be used to guide analyses. The models will be used to make quantitative predictions of behaviour and neural activity that can be contrasted with actual observations.
We will investigate: 1) simultaneous representation of multiple learning rates in dACC; 2) different reward representations in lOFC and amygdala in relation to credit assignment; 3) independent versus interactive nature of vmPFC and dACC during decision making; 4) hippocampus-dependent map-like representation to guide decision making in vmPFC and pgACC; 5) model-based versus model-free reward learning mediated by lPFC and vmPFC interaction; 6) activation and suppression of stimulus-reward representations in OFC; 7) a mechanism for determining speed, accuracy, and task engagement dependent on average reward and temporal structure of the environment; 8) distinct representations of surprise and prediction error in PFC and striatum.
Planned Impact
If awarded the grant would make it possible to conduct eight projects into different aspects of the function of the prefrontal cortex and anterior cingulate cortex. The focus is on fundamental aspects of behaviour including decision making, learning, and inference. The aim is to provide a mechanistic account of the operations of specific brain regions in relation to specific cognitive operations such as setting the rate at which behaviour changes or linking outcomes of decisions to the events and choices that caused them.
Decision making and learning mechanisms are of broad interest to many researchers working in psychology, neuroscience, and the computational modelling of behaviour. They will inform understanding and modelling of these processes.
The experiments will record neural activity in animal models using functional magnetic resonance imaging (fMRI). This will mean that the results will form a bridge between single neuron recordings made in animal models and fMRI studies (a non-invasive technique) in humans. The aim is to link specific neural signals to particular behaviour events and to understand the interrelationships between activity in prefrontal cortex and anterior cingulate cortex.
The focus is on understanding neural mechanisms in the healthy brain but the brain areas and the behavioural processes being investigated are ones that are known to be changed in many psychological illnesses such as depression. The applicants have collaborations with researcher in the Psychiatry Department, University of Oxford and related hypotheses will inform investigations in clinical populations. In addition to recording neural activity the applicants intend to measure the impact of manipulating and disrupting activity in the brain circuits they investigate. This will also enhance their ability to understand dysfunction as well as function in prefrontal and cingulate cortex.
The results will be published in peer-reviewed scientific journals and discussed at international scientific meetings that the applicants regularly attend.
The applicants have explained some of their work to non-scientific audiences at university open days and on non-specialist courses. Their research has been featured in a range of national and international radio and newspaper news articles.
Decision making and learning mechanisms are of broad interest to many researchers working in psychology, neuroscience, and the computational modelling of behaviour. They will inform understanding and modelling of these processes.
The experiments will record neural activity in animal models using functional magnetic resonance imaging (fMRI). This will mean that the results will form a bridge between single neuron recordings made in animal models and fMRI studies (a non-invasive technique) in humans. The aim is to link specific neural signals to particular behaviour events and to understand the interrelationships between activity in prefrontal cortex and anterior cingulate cortex.
The focus is on understanding neural mechanisms in the healthy brain but the brain areas and the behavioural processes being investigated are ones that are known to be changed in many psychological illnesses such as depression. The applicants have collaborations with researcher in the Psychiatry Department, University of Oxford and related hypotheses will inform investigations in clinical populations. In addition to recording neural activity the applicants intend to measure the impact of manipulating and disrupting activity in the brain circuits they investigate. This will also enhance their ability to understand dysfunction as well as function in prefrontal and cingulate cortex.
The results will be published in peer-reviewed scientific journals and discussed at international scientific meetings that the applicants regularly attend.
The applicants have explained some of their work to non-scientific audiences at university open days and on non-specialist courses. Their research has been featured in a range of national and international radio and newspaper news articles.
Publications
Ainsworth M
(2021)
Viewing Ambiguous Social Interactions Increases Functional Connectivity between Frontal and Temporal Nodes of the Social Brain.
in The Journal of neuroscience : the official journal of the Society for Neuroscience
Bongioanni A
(2021)
Activation and disruption of a neural mechanism for novel choice in monkeys.
in Nature
Chau BK
(2020)
Consistent patterns of distractor effects during decision making.
in eLife
Folloni D
(2021)
Ultrasound modulation of macaque prefrontal cortex selectively alters credit assignment-related activity and behavior.
in Science advances
Fouragnan EF
(2019)
The macaque anterior cingulate cortex translates counterfactual choice value into actual behavioral change.
in Nature neuroscience
Grohn J
(2020)
Multiple systems in macaques for tracking prediction errors and other types of surprise.
in PLoS biology
Harrison OK
(2021)
Structural and resting state functional connectivity beyond the cortex.
in NeuroImage
Khalighinejad N
(2021)
A habenula-insular circuit encodes the willingness to act.
in Nature communications
Description | Federation of European Neuroscience Societies (FENS) Committee on Animal Research Ethics (CARE) |
Geographic Reach | Europe |
Policy Influence Type | Membership of a guideline committee |
Impact | Review of EU directive 2010/63 (Animals in Science) on animal research in science |
Description | Neuromodulatory-prefrontal interactions in primates |
Amount | £4,429,363 (GBP) |
Funding ID | BB/W003392/1 |
Organisation | Biotechnology and Biological Sciences Research Council (BBSRC) |
Sector | Public |
Country | United Kingdom |
Start | 05/2022 |
End | 01/2027 |
Title | Accelerating the Evolution of Nonhuman Primate Neuroimaging. |
Description | Nonhuman primate neuroimaging is on the cusp of a transformation, much in the same way its human counterpart was in 2010, when the Human Connectome Project was launched to accelerate progress. Inspired by an open data-sharing initiative, the global community recently met and, in this article, breaks through obstacles to define its ambitions. |
Type Of Material | Model of mechanisms or symptoms - mammalian in vivo |
Year Produced | 2020 |
Provided To Others? | Yes |
Impact | only very recently released |
Title | An Open Resource for Non-human Primate Imaging. |
Description | Non-human primate neuroimaging is a rapidly growing area of research that promises to transform and scale translational and cross-species comparative neuroscience. Unfortunately, the technological and methodological advances of the past two decades have outpaced the accrual of data, which is particularly challenging given the relatively few centers that have the necessary facilities and capabilities. The PRIMatE Data Exchange (PRIME-DE) addresses this challenge by aggregating independently acquired non-human primate magnetic resonance imaging (MRI) datasets and openly sharing them via the International Neuroimaging Data-sharing Initiative (INDI). Here, we present the rationale, design, and procedures for the PRIME-DE consortium, as well as the initial release, consisting of 25 independent data collections aggregated across 22 sites (total = 217 non-human primates). We also outline the unique pitfalls and challenges that should be considered in the analysis of non-human primate MRI datasets, including providing automated quality assessment of the contributed datasets. |
Type Of Material | Model of mechanisms or symptoms - mammalian in vivo |
Year Produced | 2018 |
Provided To Others? | Yes |
Impact | Already used in a series of publication from laboratories around the world |
Title | resting state fMRI data bases |
Description | macaque resting state functional magnetic resonance imaging data sets |
Type Of Material | Biological samples |
Year Produced | 2014 |
Provided To Others? | Yes |
Impact | papers published by other research groups: Kumar V, Croxson PL, Simonyan K (2016) Structural Organization of the Laryngeal Motor Cortical Network and Its Implication for Evolution of Speech Production. J Neurosci 36:4170-4181. Mitchell DJ, Bell AH, Buckley MJ, Mitchell AS, Sallet J, Duncan J (2016) A Putative Multiple-Demand System in the Macaque Brain. J Neurosci 36:8574-8585. |
Title | Consistent patterns of distractor effects during decision making |
Description | The value of a third potential option or distractor can alter the way in which decisions are made between two other options. Two hypotheses have received empirical support: that a high value distractor improves the accuracy with which decisions between two other options are made and that it impairs accuracy. Recently, however, it has been argued that neither observation is replicable. Inspired by neuroimaging data showing that high value distractors have different impacts on prefrontal and parietal regions, we designed a dual route decision-making model that mimics the neural signals of these regions. Here we show in the dual route model and empirical data that both enhancement and impairment effects are robust phenomena but predominate in different parts of the decision space defined by the options' and the distractor's values. However, beyond these constraints, both effects co-exist under similar conditions. Moreover, both effects are robust and observable in six experiments. |
Type Of Material | Database/Collection of data |
Year Produced | 2019 |
Provided To Others? | Yes |
URL | http://datadryad.org/stash/dataset/doi:10.5061/dryad.k6djh9w3c |
Description | Distributed anatomical circuits for decision making, inference, and learning |
Organisation | University of Oxford |
Department | Department of Experimental Psychology |
Country | United Kingdom |
Sector | Academic/University |
PI Contribution | Principal Investigator on MRC grant |
Collaborator Contribution | expertise, intellectual input or the training of staff. |
Impact | Papers associated with Distributed anatomical circuits for decision making, inference, and learning grant |
Start Year | 2017 |
Description | Neuromodulatory-prefrontal interactions in primates |
Organisation | University of Oxford |
Department | Department of Experimental Psychology |
Country | United Kingdom |
Sector | Academic/University |
PI Contribution | Principal Investigator on BBSRC sLoLa grant |
Collaborator Contribution | co-applicant on BBSRC sLoLa grant |
Impact | A new award for investigating brain function |
Start Year | 2022 |
Description | Federation of European Neurosciences Socieities Committee on Animal Research and Ethics |
Form Of Engagement Activity | A formal working group, expert panel or dialogue |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | coordinating the policies of the Federation of European Neuroscience Socieities and their national affiliates in relation to animal research. Engaging with the European parliament in the review of animal research directives. Promoting openness and transparency in animal research |
Year(s) Of Engagement Activity | 2014,2015,2016,2017,2018 |
Description | Understanding Animal Research Lab Tour |
Form Of Engagement Activity | Engagement focused website, blog or social media channel |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Public/other audiences |
Results and Impact | Understanding Animal Research lab tours provides an easily accessible web resource that allows members of the general public to visit and navigate through a primate neuroscience research facility |
Year(s) Of Engagement Activity | 2017 |
URL | http://www.labanimaltour.org/ |
Description | Understanding Animal Research video |
Form Of Engagement Activity | A press release, press conference or response to a media enquiry/interview |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Public/other audiences |
Results and Impact | Understanding Animal Research (UAR) have made a video of animal research procedures used in the lab |
Year(s) Of Engagement Activity | 2023,2024 |
URL | https://www.psy.ox.ac.uk/ |
Description | interview for national news |
Form Of Engagement Activity | A press release, press conference or response to a media enquiry/interview |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Public/other audiences |
Results and Impact | Intereviews in national newspapers and scientific journals relating to ultrasound stimulation experiments (eg http://blog.pnas.org/2019/03/journal-club-low-intensity-focused-ultrasound-shows-promise-as-tool-to-probe-deep-brain-function/) |
Year(s) Of Engagement Activity | 2019 |