Integration of speech and gesture: Neurophysiological investigation

Abstracts are not currently available in GtR for all funded research. This is normally because the abstract was not required at the time of proposal submission, but may be because it included sensitive information such as personal details.

In ordinary conversation, people produce hand gestures that express meanings related to the information encoded in the concurrent speech. The listener integrates the information provided in speech and gesture to form a unified interpretation. The proposed project investigates this multimodal integration of auditory and visual information during simultaneous comprehension of speech and gesture. Four experiments with electrophysiological recordings (ERPs) and one experiment involving fMRI are proposed. The results of these experiments will not only shed light on how the brain deals with the multimodal nature of human communication, but will also further our understanding of information binding in the brain across input modalities.