MODSEM. Graded network activation and connectivity during semantic processing depending on modality
Lead Research Organisation:
University of York
Department Name: Psychology
Abstract
For any concept, we have knowledge about its characteristics and associations - for example, a rabbit is furry, has long ears and eats carrots. However, specific features of concepts are relevant in different situations, so how do we flexibly retrieve this information in an adaptive way tailored to our current circumstances? This project will test and extend a novel theoretical framework (the "graded constraints" hypothesis) which suggests that flexible semantic cognition emerges from a spectrum of large-scale networks that support more automatic to controlled patterns of conceptual retrieval, with these networks organised in a systematic fashion along the cortical surface. We will investigate how connectivity and activation along these networks varies depending on the characteristics or demands of the specific semantic processing task, and what consequences do individual differences in organisation of these networks have on semantic processing.
Organisations
Publications
| Description | The research tested the proposal that flexible semantic cognition emerges from a spectrum of large-scale networks that support more automatic to controlled patterns of conceptual retrieval, with these networks organised in a systematic fashion along the cortical surface. So far, research testing this hypothesis has exclusively examined explicit semantic decisions, but we used a more naturalistic setting, which involved reading and listening to sentences. We found that whole-brain dimensions of connectivity (also called "gradients") capture patterns of activation during naturalistic language processing, and their effects depend on the modality of presentation of stimuli (visual or auditory). Brain activation was different between spoken and written words when examining word-level characteristics (e.g., word length or frequency), but was not different for contextual or sentence-level characteristics (e.g., position of a word in the sentence). This is consistent with the view that there are aspects of language processing such as syntax or meaning that depend on more heteromodal mechanisms, and for which large-scale neural organisation does reflect how the information was presented. In contrast, other aspects of linguistic processing linked to individual words are influenced by language modality and their large-scale neural organisation reflects this. Our findings also support the use of gradients as a methodology that can help unravel the mechanisms underlying brain organisation to uncover the holistic impact of different language manipulations. |
| Exploitation Route | There might be implications for the design of deep neural network models as these start to learn from more than one modality. There might also be implications for understanding language impairments in children or following brain injury, and for rehabilitation: individual differences in the large-scale patterns associated with language processing might highlight different types of problem, allowing more effective targetting of interventions. |
| Sectors | Communities and Social Services/Policy Digital/Communication/Information Technologies (including Software) Education Healthcare |
| Title | Reading - Mind-wandering dataset |
| Description | The purpose of this dataset was to investigate how semantic knowledge helps or hinders either engaging or disengaging with information, especially when this information is in another language. We scanned 34 participants while they were presented sentences in Spanish or English, and a series of easy and hard sentences and sequences of numbers. |
| Type Of Material | Database/Collection of data |
| Year Produced | 2024 |
| Provided To Others? | No |
| Impact | Data from this study is currently being analysed by Lidón and other collaborators (Chen Chen), with promising results related to how comprehension of input modulates competition for semantic access between reading and mind-wandering, which will probably soon materialise in a manuscript for publication. |
| Description | Organiser of Pint of Science in York |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | Regional |
| Primary Audience | Public/other audiences |
| Results and Impact | Lidón organised the Psychology talks of the Pint of Science event in York, where local scientists present their results to the general public in pubs and cafés. There were two Psychology nights ("Beautiful Minds"), one called "The Speak Easy: Conversations on Language and Communication", and the other "? mind's voyage: Un??rth?ng r??lm? ?f ?m?t??n, ?l??? ?nd m?nt?l h??lth". A total of 8 speakers (PhD students, postdocs and lecturers, one of them our own collaborator Chen) talked about their research in an entertaining and engaging way. We sold out the tickets of one of the nights, and the public had many questions after the end of each one of the talks - it was very interesting to see how researchers responded to them, since they tended to be very practical and at the same time inspired others to ask their own questions. It was a great way to engage the local and regional public with research related to Psychology taking place at our university. |
| Year(s) Of Engagement Activity | 2024 |
| URL | https://pintofscience.com/ |
| Description | Science Day at St Oswald's CE Primary School in Fulford, York, UK |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | Local |
| Primary Audience | Schools |
| Results and Impact | We attended the Science Day at a local school (St Oswald's CE Primary School in Fulford, York), where we carried out activities with groups of pupils (ages 4-11). We showed the pupils how scanning the brain of a person works, the organisation of different functions in the brain, the structure of a neuron... All activities were adapted to the pupil's age (e.g., the younger ones built a colourful neuron with a simple structure using threads), and all children were really committed to the activities, asking questions and having a lot of fun. The school reported that these activities sparked their curiosity about how brains work and how we investigate them. |
| Year(s) Of Engagement Activity | 2024 |