Neuronal integration across senses: Psychophysical and computational approaches to cue integration in injured brain

This project will investigate the characteristics and neuronal underpinning for sensory integration leading to conscious experience. In particular, we are interested in interactions between visual and auditory systems leading to awareness of events. Signal Detection Theory has provided a solid ground to investigate stimulus detection across senses. Given a sensory system (e.g., audition or vision), we can quantify the level of signal that is needed (the light intensity or the sound amplitude) to be distinguished from the background and therefore to be detected. The detection capacity is then expressed as a probability function of the stimulus strength in that sense. If two signals (e.g., both light and sound) are combined, then the overall detection probability is different than that expected from each sense separately. This improved performance is termed redundancy gain.

A number of computational modelling approaches have been used to understand the mechanisms underlying redundancy gains (e.g., Otto et al., 2013). These approaches are extensions of the (unisensory) evidence accumulation framework to a scenario in which two sensory stimuli provide evidence for a perceptual decision. They use reaction time data and error rates to distil the stages of processing such as information uptake, bias, decision criteria or non-decisional components that are affected in decision making based on sensory processing. Interestingly, all the above are reliant on detection capacity and not conscious awareness.

An important discovery has been the dissociation of detection and awareness (Weiskrantz 2009). This is particularly relevant in patients with brain injury, who may be significantly above chance in detecting a signal whilst having no conscious experience of a stimulus (labelled blindsight, or deaf-hearing). Interestingly, repeated exposure to sensory stimuli can lead to increased sensitivity and recovery of function, which was observed for example after systematic stimulation using supra-threshold multi-sensory stimuli (Bolognini et al 2005). If confirmed, such multi-sensory stimulation could lead to new techniques for rehabilitation after brain injury.

To study the influence of multi-sensory input on neuronal plasticity, the proposed project is a collaborative effort combining expertise from psychophysics, computational modelling, and clinical neuroscience of the three supervisors. The project will examine audio-visual interaction in the context of reported conscious awareness using psychophysical investigation. We will develop and apply computational models in relation to reported awareness to find out which aspects of processing could be affected by multi-sensory input. In addition, we intend to recruit participants with brain injury (stroke survivors) with lesions in either primary visual or auditory cortical areas in order to examine the model predictions. These basic investigations on neuronal encoding of sensory processes are crucial in forming a solid foundation for future rehabilitation techniques.