Learning of schemas and making sense of complex events.

Lead Research Organisation: University of Sussex
Department Name: Sch of Psychology

Abstract

Where am I? Who are these people and what do they want? In our everyday lives we can usually answer these questions relatively easily, although they are by no means easy questions. To comprehend what is happening in a given situation one must perceive, maintain and integrate information from multiple modalities and timescales. Our prior knowledge can play an important role in how we comprehend and remember our surroundings. For instance, walking into a library, we might expect to see a lot of books and people studying. Indeed, our prior knowledge can help us structure newly incoming information and better understand complex events. However, this crucial aspect of our conscious experience has not been examined extensively to date, since most previous psychological research has used oversimplified stimuli (e.g. single words, pictures) that cannot possibly realistically mimic our rich and varied lived experience.
My thesis used video stimuli to act as an experimentally controlled substitute of real-life situations. I examined how different types of prior knowledge affect the comprehension and memory of complex events. We can have prior knowledge that is specific to a single situation - e.g. were we to start listening to a lecture having missed the beginning - or indeed our prior knowledge could be more schematic - such as our knowledge of how libraries typically work or how people normally behave. Schema knowledge is acquired over multiple exposures (e.g. with libraries), represents the common among episodes features (e.g. books, silence) and can have large influence on our perception. For instance, if we walk into a library, we expect to see people studying. In separate experiments I found evidence that both types of prior knowledge can have beneficial effects on learning new information.
I have produced a publication (Raykov et al., 2019) and a pre-print (Raykov et al., 2018) from my PhD work and have unpublished results for 2 research papers. The main aims of the fellowship are to prepare these results for publication and write a timely review on prior knowledge. The fellowship would also allow me to address questions left unanswered by my thesis.
My previous research has shown some of the effects of prior knowledge, however, it has not addressed how schematic knowledge develops. One of the reasons for this is that schematic knowledge is learned over multiple episodes. This means that it often is practically unfeasible to acquire data from the prolonged training period in order to investigate how individual experiences are transformed to schema knowledge.
Dr Emmanuel Barbeau, through links with Prof Chris Bird, would provide me with access to data of 30 patients, whose brain activity has been recorded (with implanted electrodes) as they watched 10 episodes of a TV show over 5 days. This would allow me to test how individuals extracted information from the multiple episodes to acquire schemas.
A specific prediction is that medial temporal lobe regions would track overlapping features among episodes to help schema development. This might happen by remembering previously learned information when encountering a situation with overlapping features. For instance, during our second visit to the library we might remember our first visit to the library in order to extract the information common across both visits (Schlichting & Preston, 2015).
The fellowship would also allow me to run a pilot behavioural project testing how we process information that contradicts our prior knowledge. These research projects will help me in the development of larger grant proposals such as ESRC New Investigator award. Furthermore, the fellowship would allow me to acquire new research skills and organise public engagement event to disseminate my research to a wider audience.

Publications

10 25 50
 
Description I have ran multiple behavioural experiments online that address how specific features of an event (e.g. a video describing a situation) can affect false memory for the event later on. For instance, we have pre-registered and replicated in multiple experiments that participants are much more likely to falsely remember the end of an event that was interrupted before participants saw a completed action. For example, if a baseball video cuts before participants see the bats man hitting or missing the ball, participants are quite likely to falsely remember that the video showed an outcome a week later. So later on they may remember that they saw the batter hitting a home run, even if that was never shown. Such addition of false details was not seen if the video ended after showing the batter hitting the ball but stopping while the ball was in the air. We have submitted this collection of experiments for publication and it is currently under second round of reviews. We feel this new behavioural paradigm can be important to address important questions such as what types of prior knowledge contribute to false memories. We have started a collaboration with a group from Italy where we will test how these false memories are exhibited in older adults and adults with brain damage in their pre-frontal cortex.
Furthermore, with collaboration with researchers from Sussex, we ran two fMRI studies that aimed to investigate how we process information that is incongruent with our prior knowledge. Participants watched videos that included actions that did not match the context, were surprising and inappropriate. This extends my PhD work that examined how prior knowledge can benefit learning. With this experiment we will test how prior knowledge affects learning of incongruent information. Furthermore, there are strong theoretical predictions about the brain regions that may support prior knowledge. We pre-registered our hypotheses about both of these fMRI experiments. Using two experiments we could distinguish whether current predictions about a situation came from general prior knowledge, or semantic memory, or were based on specific prior knowledge, an episodic memory of having seen a different version of the same video. This allowed us to address important theoretical questions and our findings contradict standard views on what the hippocampus does for memory. The hippocampus is a very important region for learning new information and is often affected in the early stages of dementia. We found that despite common views the hippocampus is not strongly involved in making online predictions based on semantic knowledge. In contrast it seemed the hippocampus is particularly responding to information that does not match our prior memories. For instance, if we re-watch a video and one of the actions is different the hippocampus would strongly respond only to the different action. This implies that unlike previous theories hippocampus is not necessarily a general prediction machine but only responds to episodic prediction errors. Indeed, this is inline with views that hippocampus is particularly engaged when we have to remember information in times of uncertainty. Apart from providing evidence challenging widely held views about the hippocampus and broadly accepted cognitive theories, our results also show that unlike some recent suggestions participants are not particularly good at learning information that is incongruent with their prior knowledge. By using naturalistic videos we showed that memory for incongruent actions was worse when participants had to freely recall the information. This is in contrast to previous findings using single words or picture stimuli and testing memory with recognition paradigms.
Exploitation Route Our outcomes inform and extend theories of event segmentation, prior knowledge and predictive coding. Furthermore, they can inform theories about false memories. Furthermore, our findings can have applied impact as they can be used to show that whether when learning new information is better when the new information is congruent or incongruent with prior knowledge.
Sectors Education,Healthcare

 
Title Adaptive latent feature sharing for piecewise linear dimensionality reduction 
Description It is very common to use some type of dimensionality reduction when working with large datasets. For example, if we have a collection of measurements (blood pressure, IQ test, memory test, Exercise Activity, Social economic status etc.) for each participant we may want to examine the relationships among these variables effectively. This low-dimensionality embedding is common also in neuroimaging where we might have large number of measurements (a typical brain dataset may include 100 000 voxels - 3D pixel- collected every 1 second over the course of an 1 hour or more). There have been multiple algorithms developed that try and project the data into low-dimensional space or basically examine the relationships among variables in the dataset. This has most commonly done using principled component analysis or factor analysis which in their most simple form rely on the correlations among variables. They are widely used in the field although they may be sensitive to outliers and fail to capture the truly most important relationships in the data. Recently there has been development of non-linear low-dimensionality embeddings, which can provide interesting results, but have also been difficult to interpret or criticized for giving false positives. Here in collaboration with researchers from Aston and Nottingham University, we developed a novel approach - adaptive PCA - that can use linear project to solve non-linear dimensionality reduction problems. As such it can provide more interpretable results, is more robust to outliers and can accurately 'adapt' to the most important variables relationships in the dataset. The model has been accepted for publication in the journal of Machine learning research and we have developed a tutorial notebook on how to apply the method to different types of datasets. 
Type Of Material Data analysis technique 
Year Produced 2023 
Provided To Others? Yes  
Impact This work has only been recently developed, we have showed that this low-dimensionality embedding is more robust and interpretable than commonly used methods that either are not flexible enough to capture the true complexity in the data or are too flexible and overfit to noise in the dataset. 
URL https://arxiv.org/abs/2006.12369
 
Description False Memories for event endings 
Organisation University of Bologna
Country Italy 
Sector Academic/University 
PI Contribution We have ran multiple pre-registered experiments where we showed that participants commonly have false memories about how events ended. After developing this paradigm I sent the experiment to a group in Italy, which will test it with older adults and adults who had suffered stroke. Specifically, they will examine whether individuals with brain damage in pre-frontal cortex would paradoxically show no false memories compared to healthy older adults. This can inform neurobiological theories on how pre-frontal cortex control behaviour and stores prior knowledge.
Collaborator Contribution The collaborator is a prominent researcher in the field of psychology. They has access to a highly sought out group of stroke patients that have specific brain damage. These patients are very rare and very useful for psychology research as they can provide causal evidence of the involvement of pre-frontal cortex for performance on certain tasks. The collaborator has had multiple high-impact publications illuminating how patients with ventromedial prefrontal cortex perform on multiple tasks.
Impact This is a collaboration that recently started so we do not have outputs yet. I am in the field of cognitive neuroscience. The collaborator is in the field of psychology and neuropsychology.
Start Year 2023
 
Description Latent feature sharing: an adaptive approach to linear decomposition models 
Organisation Aston University
Department School of Engineering and Applied Science
Country United Kingdom 
Sector Academic/University 
PI Contribution I have tested a newly developed algorithm for performing dimensionality reduction. Specifically, a new method to perform PCA. I was involved in adjusting the algorithm to work with fMRI data and testing its performance on fMRI data compared to more standard approaches of dimensionality reduction. I was involved in the write up of the paper describing the testing of the algorithm on the fMRI datasets and involved in the writing of interactive tutorial code notebook that aims to explained the benefits of using the new algorithm and show how to use it.
Collaborator Contribution My contributors were involved in both the conceptual and theoretical development of the algorithm and also in testing the performance of the algorithm on standard datasets, widely used for testing of dimensionality reduction algorithms in the machine learning field.
Impact The article received positive reviews and was accepted for publication in the Journal of Machine Learning Research. We are currently doing final edits for the article. The article is available as pre-print https://arxiv.org/abs/2006.12369; There is also an associated tutorial notebook that we have been developing that can be run in the browser. This will be released with the publication of the paper. This is a multi-disciplinary collaboration. My collaborators are working in the field of Statistics and Machine learning. I work in the field of Cognitive Neuroscience and Psychology.
Start Year 2020
 
Description Learning of schemas and making sense of complex events 
Organisation University of Toulouse
Country France 
Sector Academic/University 
PI Contribution I am working on data collected at the University of Toulouse from patients undergoing Epilepsy treatment. They have implanted electrodes in their brains which can allow us to better measure brain activity.
Collaborator Contribution My contributed has kindly shared the data from the patients and provides me with guidance in analysing this dataset.
Impact The planned projects were investigated and the collaboration finished. There were no publications that resulted from this collaboration. I work in the field of Cognitive Neuroscience and Psychology, similarly to my collaborator.
Start Year 2020
 
Description Blog on my research 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact I wrote a blog post summarising my research and the state-of-the art methods and findings in my research field. The post is suitable for lay person and gives intuition on the important research questions in my field.
Year(s) Of Engagement Activity 2022
URL https://blogs.sussex.ac.uk/psychology/2022/03/02/how-does-prior-knowledge-affect-learning-of-new-inf...
 
Description Science Presentation at House of Commons 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact I presented my research at an event aimed at celebrating 60 years of Sussex University. The event was held at the house of commons and I could present my work to invited politicians and policy makers. The discussion sparked questions both about my research, my future ideas, and potential impact of our work.
Year(s) Of Engagement Activity 2021