Beating auditory beats: plasticity and selectivity in the multimodal integration of cues to the temporal control of action

Skilled human movement and social interaction depends on our ability to synchronise our actions with those of other people or objects in our surrounding environment. For example, when we dance with a partner we use auditory information from the music played, tactile information from our partner, proprioceptive information from the position of our limbs in space and visual information from flashing lights, to keep our movements on the beat. Previous research examining the way in which humans produce synchronised movements has focused predominantly on the use of auditory information (i.e. regular auditory beats). This proposal considers human performance in situations more like everyday life where we use more than one source of sensory information to perform a rhythmic movement. A major challenge in understanding performance in this multi-modal setting is to discover how the brain evaluates the relative importance of each piece of information it receives. Audition, vision and touch can all provide information (cues) to a beat, but which cue should we use? A simple strategy might suggest only the 'best' source of information, but no simple rule determines the circumstances in which a piece of information is most appropriate. Intuitively, it seems sensible to use all the information available, but when doing so to give more importance (or weight) to the source of information that is most reliable. This intuition can be shown mathematically, and it is known that optimal performance in a multi-modal setting results from using all the information available weighted according to the reliability of each cue. This proposal investigates the combination of multi-modal information for the temporal control of action, studying the brain's use of auditory, tactile and visual information when controlling the rhythmic movement of our limbs. We will test and develop theoretical models for the combination of different sensory signals and compare these models with human performance. We will evaluate how the brain makes use of the statistics of sensory signals so that it can evaluate the reliability of timing cues and determine the extent to which human timing behaviour reflects the best use of the available information. We will study the dynamics of the combination process to determine the brain's plasticity in responding to changes in the reliability of particular sources of sensory information. Finally, we will examine age related changes in the use and statistical assessment of sensory cues to timed action, to evaluate how the changes in the processing capacities of our sensory systems impact on our ability to produce successful timed action. The proposed research will study human behaviour to develop understanding of sensory integration for motor timing. It will provide a basis for future work examining the underlying neural mechanisms using brain imaging techniques. The work has implications for the development of multimodal metronomes for music performance or as teaching aids and as such has potential for commercial development. There are also implications for healthcare with possible applications in movement rehabilitation or elderly exercise routines based on improvements in metronome assisted retraining of motor function.

Adaptive human behaviour depends on the brain detecting and responding to the statistical regularities of the environment to maximise its use of available information. A skill fundamental to successful human behaviour is the ability to synchronise actions with environmental events / ensuring that we are in the right place at the right time. We propose a three-part programme of behavioural experiments to examine the brain's use of multi-modal sensory signals in the temporal control of action. We will study the production of timed movements (finger taps) and analyse the statistics of observers' responses (central tendency and dispersion of the asynchrony between the beat and taps). Phase 1 of the experimental programme examines the extent to which the control of finger tapping reflects the Maximum Likelihood Estimate of tempo derived from the combination of auditory, visual and haptic cues to beat. Phase 2 examines the brain's estimation of the statistics of sensory signals, an important step in determining the weight given to a signal during combination. First, we ask what constitutes noise in the estimation of signal reliability, determining whether the brain is able to disregard an irrelevant noise source when estimating signal weight. Second, we ask whether the brain exploits space-time correlations to compensate for the limitations of a particular source of information. Phase 3 examines the dynamics of signal estimation. First, we consider short-term dynamics, examining how observers' tapping responses change in response to changed stimulus reliability. Second, we examine age-related changes that reflect the brain's long-term recalibration in response to changes in processing dynamics of the sensory apparatus. The work will provide novel insight into the temporal control of action, the brain's estimation of signal statistics and sensory processing in older adults. It has potential application in the fields of training and rehabilitation.