A quantitative framework of attentional selection

Lead Research Organisation: University of Aberdeen
Department Name: Psychology

Abstract

Attentional selection is an essential prerequisite for adaptive behaviour. The brain receives a vast amount of information from our sensory organs. Fully processing all of this information would exceed the capacity of the human brain, therefore attention selects only some of this information for in-depth processing at the expense of other, less relevant information. Attention is a key concept in the understanding of mental abilities: it critically influences perception, memory, cognition, emotion and consciousness, so that what we see, remember, think, feel and experience is, to a large extent, a function of selective attention.

Selectively attending to a stimulus enhances its processing in visual cortex and increases accuracy and speed of motor responses. While the connection between these findings seems obvious, our understanding of this connection is surprisingly vague: we know that, in the average of many trials, measures of cortical processing and behavioural performance are enhanced when the eliciting stimulus is attended, but we have very limited understanding of how these attentional effects are quantitatively related.

The proposed project aims to develop a quantitative model of visual attention that directly links electrophysiological measures of stimulus processing in visual cortex with behavioural outcomes. This will be achieved in a sequence of three connected objectives:

First, we will examine how concurrent attentional selection of different dimensions (e.g. location, colour, orientation or motion) affects stimulus processing. This will allow us to better understand attentional selection in complex realistic situations and facilitates quantitative modelling in the subsequent experiments by restricting model parameters.

Second, we will conduct a series of three EEG experiments in order to uncover the quantitative connections between attentional modulation of stimulus processing in visual cortex and behavioural measures. This will significantly enhance our understanding of how attention supports adaptive behaviour.

Third, we will investigate how monitoring of behavioural performance is used to readjust attentional model parameters over time in order to optimise subsequent performance. This will help us understand why performance on the same task fluctuates over time. It also provides important information on why attentional model parameters take on specific values.

The answers to these questions fundamentally contribute to our understanding of visual attention and perception. This research may have implications for research in special populations (e.g. children and older participants, patients with attentional disorders) as the experimental protocols and modelling approaches of the present project could be applied to gain a better understanding of group differences. Additionally, this work may facilitate the further development of Brain Computer Interfaces (BCIs), which allow subjects to convey their intentions by shifting attention and which commonly employ similar technical approaches as the present project.

Technical Summary

The brain's capacity for visual processing is limited. Hence, selective attention directs processing resources to relevant stimuli in order to allow for adaptive behaviour. Previous research has documented attentional effects on measures of neuronal activity and behavioural performance, however these different types of measures have rarely been linked quantitatively.
Existing quantitative models of visual attention currently do not allow us to make direct connections between behavioural and neuronal measures. Although Bundesen's Theory of Visual Attention (TVA) has recently been reinterpreted in the context of neuroscientific models of attention, its empirical foundations still remain behavioural. The biased competition, feature-similarity gain and normalization models on the other hand provide quantitative descriptions of attentional effects on individual neurons, however they do not offer a direct quantitative link to behavioural data.
The proposed project aims to develop a quantitative model of visual attention that directly links electrophysiological recordings of 'frequency-tagged' steady-state visual evoked potentials (SSVEPs) with behavioural outcomes. This will be achieved through three connected objectives:
First, we will examine how concurrent attentional selection of different dimensions (e.g. location and colour) affects stimulus processing. This will facilitate modelling in the subsequent experiments and helps understand attentional selection in complex realistic situations.
Second, we will conduct three EEG experiments in order to uncover quantitatively how attentional modulation of stimulus processing in visual cortex affects behavioural measures.
Third, we will investigate how performance monitoring leads to readjustment of attentional model parameters. This helps us understand the basis of fluctuations in performance and also yields information on why attentional model parameters take on specific values.

Planned Impact

The primary impact of the proposed research will be an increased understanding of fundamental aspects of human selective visual attention. Selective processing of sensory inputs plays a critical role in how we experience the world around us; understanding it is intrinsically valuable.
Due to the fundamental nature of this research, the most immediate beneficiaries will be other researchers of three types: (a) researchers who are interested in integrating the results with other findings in order to develop a more complete functional understanding of the human brain (b) researchers who are interested in applying the 'frequency tagging' technique to investigate other cognitive phenomena and (c) researchers who may apply the results to investigate attentional disorders or cognitive ageing or to further the development of Brain Computer Interfaces (BCIs), which allow subjects to convey their intentions to a computer by shifting attention between stimuli (e.g. letters or digits) in a visual display. In order to reach these researchers, the work will be submitted to high-impact journals with a large readership and presented at various national and international conferences with wide audiences. Particular care will be taken to present the findings and possible implications in nontechnical language to the media to make them accessible to non-specialist audiences.
This project will also provide rigorous scientific research training to a post-doctoral research assistant, who will develop specialized skills such as the utilization of psychophysical and electrophysiological techniques, time-frequency analysis of biological signals and more general skills such as programming and scientific writing.
Finally, the United Kingdom's reputation as a leading country for fundamental research with highly influential contributions in the fields of cognitive and visual neurosciences will be reinforced through the advances afforded by the proposed research. Publishing the results in top journals will attract more excellent researchers to work at UK institutions and thereby advance the scientific community here.
 
Description This project consisted of a series of experiments where steady-state visual evoked potentials (SSVEPs) were used to continuously measure the allocation of attention in early visual cortex.

Discovery 1. Observers rapidly adjust attentional parameters based on internal detection of behavioural errors.
In one of the studies we recorded SSVEPs elicited by continuously presented random dot stimuli while participants were performing a challenging attentional task. We found that attentional selectivity was reduced just before error trials, and that once the error was made it was immediately reallocated towards task-relevant stimuli. These results reveal one of the systematic sources of between-trial variations in attentional selection - a continuously engaged error monitoring system.
Discovery 2. Spatial and feature-based attention independently influence early visual cortex.
Two experiments investigated how feature-based attention operates together with other types of attention: spatial and object-based. In the first experiment we hypothesised that if attentional selection of features and space rely in part on a shared resource, then attentional selection of colour would be less efficient when spatial attention is divided between two locations. We measured SSVEP amplitudes while participants were cued to attend the dots of a certain colour either on one side or on both sides of the screen and found that the amplitudes were independently modulated by spatial and colour-based attention. In the second experiment, we presented participants with two fully overlapping counter-rotating bicolored surfaces of light and dark dots and asked them to attend to a subset of dots defined by feature (luminance) and object (surface). In line with the results of Experiment 1, SSVEPs elicited from early visual cortex exhibited independent multiplicative enhancement by feature- and object-based attention.
Discovery 3. SSVEPs may be used to predict behavioural outcomes in attentional tasks.
In two experiments SSVEP recordings were combined with modelling of behavioural responses. In both experiments we analysed SSVEP amplitudes preceding correct responses and manipulated feature-based attention, but the nature of the attention tasks was different in each case. Correlational analysis of reaction times and SSVEP amplitudes in both studies shows that larger attention-related modulations of SSVEPs are linked to larger changes in reaction times. To further pinpoint the source of this correspondence behavioural responses were modelled using a hierarchical Bayesian framework. The results show that consistently across the tasks focused attention results in faster accumulation of information. This rate of the increase in the parameter describing evidence accumulation corresponds to the rate of increase in SSVEP amplitudes, suggesting that SSVEPs are predictive of behavioural outcomes.
Method development. In order to determine whether different features in attentional selection are combined in an additive or in a multiplicative manner, we carried out a re-analysis of eight existing EEG datasets. We used meta-analytic approach combined with mathematical modelling to show that the data are best described by multiplicative interaction of attentional factors. We also used the same dataset to quantify the effect of attentional selection on the phase of SSVEPs and found that attention speeds up SSVEP latencies by 5-20 ms depending on the condition.
Exploitation Route Academically, the outcomes of the project can spark a number of further investigations. An existing collaboration is studying the relationship between reward and attentional selection in a similar approach as our study on performance monitoring (Discovery 1). The link between SSVEP amplitudes and behavioural performance (Discovery 2) can be further studied in context of individual differences (e.g. whether it is possible to train attentional control and achieve behavioural improvements using biofeedback approaches).
Outside basic cognitive research, the enhanced understanding of basic attentional mechanisms can be used to facilitate the development of brain-computer interfaces (BCIs), which currently mostly rely only on spatial-based attention. BCIs commonly employ the SSVEP technique in order to allow users to convey their intentions to a computer by shifting attention between stimuli in a visual display (i.e. without the use of classic input tools such as a mouse or a keyboard). Our results establish relative independence of various types of attention (e.g. spatial and feature-based), which means that it is feasible to track them concurrently and achieve faster and more reliable readout from a BCI setup.
Sectors Digital/Communication/Information Technologies (including Software)

Electronics

 
Description Attention and camouflage in three dimensions
Amount £89,000 (GBP)
Funding ID 2096492 
Organisation Biotechnology and Biological Sciences Research Council (BBSRC) 
Sector Public
Country United Kingdom
Start 09/2018 
End 09/2023
 
Description Dealing with irrelevant information: An inter-domain examination of visual perception and cognitive psychology approaches
Amount £75,000 (GBP)
Organisation Carnegie Trust 
Sector Charity/Non Profit
Country United Kingdom
Start 09/2019 
End 09/2022
 
Description Development Trust: How do we keep track of what is where: The electrophysiology of multiple identity tracking.
Amount £1,000 (GBP)
Organisation University of Aberdeen 
Sector Academic/University
Country United Kingdom
Start 06/2018 
End 08/2018
 
Description Development Trust: Sustained effects of conflict monitoring on feature-based visual attention
Amount £200 (GBP)
Organisation University of Aberdeen 
Sector Academic/University
Country United Kingdom
Start 06/2018 
End 08/2018
 
Description Electrophysiological Assessment of Visual Processing After Brain Injury
Amount £89,000 (GBP)
Funding ID 1959791 
Organisation Biotechnology and Biological Sciences Research Council (BBSRC) 
Sector Public
Country United Kingdom
Start 09/2017 
End 09/2021
 
Description Leverhulme Early Career Fellowship
Amount £93,000 (GBP)
Funding ID ECF-2020-488 
Organisation The Leverhulme Trust 
Sector Charity/Non Profit
Country United Kingdom
Start 09/2020 
End 10/2023
 
Title Combined data from 9 SSVEP studies of visual attention 
Description The database combines data from 9 published SSVEP studies of visual attention. The dataset is openly available from OSF platform (https://osf.io/u8qts/), was assigned a DOI (10.17605/OSF.IO/U8QTS) and was described in detail in the paper "Attentional Modulation in Early Visual Cortex: A Focused Reanalysis of Steady-state Visual Evoked Potential Studies". 
Type Of Material Database/Collection of data 
Year Produced 2023 
Provided To Others? Yes  
Impact The dataset allowed to carry out novel analyses published in the associated paper "Attentional Modulation in Early Visual Cortex: A Focused Reanalysis of Steady-state Visual Evoked Potential Studies". The dataset will benefit future SSVEP studies of attention by providing benchmark effect sizes and parameters for data simulation. 
URL https://osf.io/u8qts/
 
Description CAPLabGent 
Organisation University of Ghent
Country Belgium 
Sector Academic/University 
PI Contribution Expertise in theories of attention. Technical expertise in SSVEP conducting SSVEP experiments, including time-frequency analysis. Provided custom written Matlab code for running experiment and analysing data. Experimental design.
Collaborator Contribution Experimental design, data collection, data analysis
Impact Results published in paper.
Start Year 2017
 
Description CogCont 
Organisation Catholic University of Eichstätt-Ingolstadt
Country Germany 
Sector Academic/University 
PI Contribution Expertise in theories of attention. Technical expertise in conducting SSVEP experiments, including data analysis. Training of staff in these techniques (1 PhD student, 1 PostDoc). Access to laboratory at University of Aberdeen. Custom-written Matlab codes for conducing experiments and analysing data.
Collaborator Contribution Expertise in theories of cognitive control. Data collection at Katholische Universität Eichstätt-Ingolstadt. A total of 4000€ to allow a postdoc and a PhD student to visit my lab for two months to conduct experiments and gain experience here ("Förderlinie 3 der internen Forschungsförderung proFOR+ der Katholischen Universität Eichstätt-Ingolstadt", Grant U995100103: 2000€; "internal grant for a lab visit within the Priority Project 1772 of the German Research Foundation: 2000€).
Impact Disciplines: Cognitive Neuroscience / Cognitive Psychology
Start Year 2016
 
Description Collaboration on Quantitative Modelling with Center for Visual Cognititon, University of Copenhagen 
Organisation University of Copenhagen
Country Denmark 
Sector Academic/University 
PI Contribution - Intellectual input in the form of overall guidance - Provision of more specific expertise with respect to implementation. - Access to data
Collaborator Contribution Intellectual input into approach to quantitative modelling of behavioral data and hands-on application of expertise in implementing this.
Impact In progress...
Start Year 2022
 
Description EEGManyLabs 
Organisation Eberhard Karls University of Tübingen
Country Germany 
Sector Academic/University 
PI Contribution Expertise in EEG data analysis. Data extraction from pre-identified studies. Custom R scripts for data simulation and power analysis.
Collaborator Contribution Systematic search for published EEG studies in databases, screening of studies for eligibility.
Impact I was invited to present at the EEG data analysis workshop in Saint Petersburg, Russia (described under Engagement Activities).
Start Year 2019
 
Description Article in Psychology Newsletter 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Undergraduate students
Results and Impact I wrote a one page article about the awarded grant project for the Psychology Newsletter. The publication has led to increased interest in the research topic by colleagues and students.
Year(s) Of Engagement Activity 2016
URL https://www.abdn.ac.uk/psychology/documents/Newsletter_Winter_2016.pdf
 
Description Data analysis workshop (Saint Petersburg) 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Postgraduate students
Results and Impact Invited instructor at a residential workshop in Saint Petersburg, Russia. 50 postgraduate and undergraduate students participated in a series of interactive classes on EEG analysis using existing open datasets. The activity sparked discussion around open data culture and future multi-lab collaborations such as EEGManyLabs.
Year(s) Of Engagement Activity 2019
 
Description May Festival activity 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact We held a drop-in session about the studies of attention at May Festival (science engagement event at the University of Aberdeen) . We showed demonstrations of visual search and multiple objects tracking experiments and collected pilot data from the members of public aged 4-40. Pilot data will be used to set up future studies on age differences in attention and working memory. We also answered questions about the methodology and potential applications of EEG research and cognitive studies in general.
Year(s) Of Engagement Activity 2018
 
Description Poster Presentation at the Neuroscience Academy Denmark (NAD) event at University of Southern Denmark 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Other audiences
Results and Impact This multidisciplinary event included researchers, practitioners, and prospective PhD students in the field of neuroscience. The purpose of the event was to inform prospective PhD students of opportunities for labrotations and to foster interdisciplinary and translational research. 60-70 attendees came to see and discuss my poster and some potential options for new collaborations were identified.

The NAD PhD programme was founded only a year ago and is still in development. The programme coordinator showed great interest in my experience with the EastBio BBSRC DTP in the UK and in learning more about the well-developed practices of BBSRC DTPs.
Year(s) Of Engagement Activity 2024
URL https://neuroscienceacademydenmark.dk/
 
Description Talk (postdoc appreciation week) 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Postgraduate students
Results and Impact I talked about the results of the project to the audience of researchers and postgraduate students of the University of Aberdeen. I received quite a few follow-up questions and discussed a potential cross-disciplinary collaboration with a colleague from the Department of Economics.
Year(s) Of Engagement Activity 2020
 
Description Talk at the Icelandic Vision Group seminar 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Postgraduate students
Results and Impact I talked about the results of the project to the audience of researchers and postgraduate students at the University of Iceland. The methodology of mini-metaanalysis sparked a lot of interest.
Year(s) Of Engagement Activity 2024