Brain processes predicting future perception: cortical feedback and visual predictions

Brain processes predicting future perception: cortical feedback and visual predictions We investigate cortical processing strategies using functional brain imaging. Previous evidence has shown that early visual areas receive feedback from higher visual areas. We have shown previously that this feedback activity helps to predict future perception. In three functional magnetic resonance imaging (fMRI) experiments we will test feedback and prediction with natural visual scenes (experiment 1 and 2) or with natural viewing conditions (experiment 3). To extract meaning the brain has to set incoming sensory information into the context of previous and future events. From moment to moment the visual system needs to decide what is present in its environment and how to interact with it. Perception 'postdicts' events if they occur unforeseen, that is perception is a reconstruction of the recent past, if upcoming events are however predictable the visual system incorporates this knowledge in the perceptual processing. Little is known about the brain processes underlying this predictive coding. One of today's biggest scientific challenges is the identification of important cortical processing principles. Tentative models that try to identify the essential principles are relatively rare. Feedback and prediction play an important role in some recent theories about brain processing. The memory-prediction framework of intelligence has been a relatively recent approach to isolate essential processing strategies. In this theory, the main tasks of cortical modules are to extract a plausible categorisation that helps to predict future events. Cortical areas integrate upcoming and neighbouring information and forward the current interpretation to higher areas and at the same time back-project the results to lower tier areas. Lower cortical areas compare the upcoming information with the prediction of higher cortical areas and generate a new pattern or error signal if the prediction was incorrect. The memory prediction framework assumes that the processing principles are quite general and apply to all cortical areas including primary sensory areas. Some evidence in favour of feedback and prediction has used artificial / easy to predict / stimuli. Here we will test in primary visual cortex (area V1) whether complex predictions can be formed with natural visual scenes. Two experiments will use complex visual stimuli taken from 3D-movies to initiate complex expectations. Subsequently the predictions will be tested using a newly developed fMRI-anticipation design. The third experiment will test visual predictions in the context of eye-movements.

This proposal is about cortical feedback and visual prediction. In recent years we have shown that V1 receives feedback from specialised visual motion areas (Muckli et al. 2005, PLoS) and that this feedback helps to predict future perception (Schwiedrzik et al. 2007, VisRes). With this grant we aim to investigate cortical feedback and visual prediction in natural visual scenes. The central importance of feedback and prediction for cortical processing has been highlighted recently by the memory-prediction framework described in the book 'On Intelligence' (Hawkins & Blakeslee, 2004). We propose three new experiments that will test hypotheses derived from the memory-prediction framework. In the first experiment we will record rapid event related fMRI while natural 3D-images are presented in correct or scrambled temporal sequences. Presentation in the correct temporal sequence will induce complex apparent motion and should trigger predictive coding. Such predictions will be tested with a probe stimulus for which we expect to see BOLD signal attenuation if it correctly fits with the prediction. The second experiment will use multivariate analysis strategies to extract information from non-stimulated subregions of area V1. We will use 3D-pictures of natural scenes and place an occluder on top of the image. Following our predictions about cortical feedback and visual prediction and the memory-prediction framework we propose that this non-stimulated cortical region might carry significant informationby means of feedback or lateral interactions. The third experiment will use our previous long range apparent motion paradigm and combine it with eye-movements. In this experiment we will test whether pre-saccadic perceptual expectations will be fed-back to V1 to represent the new retinal positions of the post-saccadic stimulation scene. With these three experiments we will be able to test whether cortical feedback codes for perceptual expectations in natural scenes.