Integrating perception and action: the multi-channel model of visuo-motor control

When a skilled cricket player reaches out to catch a ball, multiple brain systems simultaneously predict the position of the ball and the arm. Based on this visual information, the motor system must rapidly correct the ongoing movement, driving the hand in a manner that ensures successful grasp of the ball. Similar visual and motor pathways are used in all sports, where an athlete is required to respond as quickly and accurately as possible to the small changes in the world signalled by visual feedback. Indeed, these processes are fundamental to most skilled movements in everyday life.
The visually sensed changes may refer either to one's own body, or to objects in the external world (such as a ball). By assigning these visual changes to either itself or to an external item, the visuo-motor control system can respond quickly in the correct manner to ensure skilled action. In this project, we will investigate the neural pathways that constitute this vision to motor action loop. Specifically, we will study three major questions:
The first question is how these different pathways, signalling either our hand location or the target location (e.g. ball), interact with one another. The current scientific view is that the brain simply calculates the difference between the target and hand locations and uses this difference to correct the movement. However, our preliminary experiments demonstrate that this is not true, but rather suggest that the two pathways lead to partially independent responses. Using a robotic device, we will carefully measure the interactions and independence of these two feedback pathways.
The second question investigates which parts of the brain are dedicated to the processing of the two feedback pathways. We will investigate active reaching movements using a robotic device while measuring brain activity using functional magnetic resonance imaging. The individual activity pattern in each region will reveal how target and hand information are represented in different brain regions, and how these regions interact.
The third question is how the brain assigns visual signals to one's own movements, or other action-relevant objects. For example, a huge number of sports utilize bats, or rackets that act as an extension of the subjects own hand. The brain must therefore assign agency to these objects, marking them as self, in order to respond correctly to visual changes in these objects, which may be a different action than responding to changes in external objects such as the ball. We will investigate the process by which this occurs and attempt to distinguish it from attention mechanisms.
This project investigates the basic vision to motor action pathways that underlie skilled movements. Understanding these pathways and the manner in which the brain utilizes them for fast action will lead to improvements of training regimes for high-performance sports. In many sports, the highest level of performance requires the ability to respond accurately and with exceptional speed to small, barely detectible visual information.
The research also produces an essential understanding of the pathways directly involved in learning of action. As such, it provides important information on the mechanisms used in learning and retraining skills and movements. This has particular relevance for rehabilitation after brain injury, such as stroke. Extensive techniques are being developed which use robotic devices for retraining after brain injury, where feedback is also provided visually. By understanding in detail how and where this visual information is processed, optimal training designs for stroke rehabilitation can be developed, which take into account individual deficits in the various feedback loops.

This research investigates the visuo-motor control system responsible for online responses to changes in the visual field. The classical view of this process is that the visuo-motor system computes the difference vector between the hand and target, and sends this to the motor system to compute the necessary corrective commands. However, our recent data suggests that this view of online visuo-motor control may be incorrect. Instead, it appears that the information of target motion and hand motion reach the motor system over separate channels, which only partially interact with one another. Over a series of experiments, we will develop and comprehensively test a new model of visual feedback control. First, we will test whether, and to what degree, these feedback pathways are independent during online control, fully characterising their interactions. To further explore this issue, we will examine whether the gains of these systems can be independently controlled. Using functional MRI, we will then examine the neural correlates of the feedback channels in the human brain. We will investigate the encoding patterns of the whole brain in regards to these visuomotor signals with multivoxel pattern analysis. Finally, we will investigate the relationship between visual attention and the assignment process of visual information to the separate feedback channels. The studies proposed here provide a novel view on visual feedback control. Breaking away from the classical division of perception, decision and action processes, this research will provide a model encompassing all stages of the perception-action loop. By studying the neural underpinnings of this process, we will be able to shed new light on the functional organization of visuo-motor regions, especially those of the parietal cortex. Firmly establishing the basic processes that enable fast and automatic online control of action is the foundation for understanding how the system achieves flexibility of control.

This research will benefit multiple groups over the next decade. Initially, the biggest impact will be for basic researchers investigating neural control, attention, computational neuroscience, sport, and rehabilitation science. However, the knowledge gained of both the structure and mechanisms of visuo-motor control will affect both high-performance sport and rehabilitation, particularly from stroke injury. While this research aims to thoroughly investigate the foundation of the neural control of movement, in the long term, this work should also provide benefit for both the UK sporting agencies (improving training regimes), and the general public health (improving physical rehabilitation techniques).
Our work investigates the neural underpinnings of visuo-motor control. In particular, the interaction between the visual and motor systems involved in executing rapid, accurate motor responses to visual information. This provides basic understanding of the mechanisms of vision to motor action critical for sporting success. In many sports, optimal performance is only achieved through rapid accurate corrective responses to small changes in the complex visual environment. For example, a batter must respond to tiny changes in both the ball's flight path and rotation in order to make contact. However, the neural mechanisms through which the brain performs these tasks is still unknown. In this project, using a variety of techniques, we will pin down both the neural pathways and the computational algorithms necessary for these exceptional visuo-motor acts. Only through the understanding of these mechanisms can specialized training be designed to optimize the speed and accuracy of the visual feedback pathways, which can be responsible for the infinitesimal differences between wining and losing.
The third project aim investigates the assignment of agency to particular signals in the visual information. Certain visual signals (such as the controlled wooden bat) must be assigned to oneself, while other visual signals must be assigned either to target objects (such as the ball) or to signals which may affect overall strategy (such as the positioning of opposing players). As soon as any one of these visual stimuli changes, the visuo-motor system must take appropriate (but normally very different) action to ensure optimal performance. We will explore this assignment process and its interaction with the focus of attention. These findings have implications for instructing and training players to focus on particular features of their environment. Thus, our work will provide a detailed scientific basis of optimal focus for sport performance.
This research will not only have applications in high performance sport, but also provides critical understanding of the neural pathways directly responsible for some aspects of motor learning and action. Thus, this project will provide essential understanding of some of the neural mechanisms involved in motor learning, and therefore in (re)training. This has particular relevance for neural rehabilitation after brain injury (such as stroke, which affects 150,000 people every year in the UK alone). Recently, many research laboratories are developing rehabilitation techniques using robotic devices to provide haptic and visual feedback. The use of robotic devices for rehabilitation is in many ways ideal as they can provide appropriate levels of assistance, guidance, or even resistance depending on the abilities of the individual patient. However, they also allow control over the visual and haptic feedback provided to the patients, which is only marginally exploited so far. By understanding how and where this visuo-motor feedback is processed, optimal training designs for stroke rehabilitation can be developed by taking into account individual deficits and / or preserved function of the various feedback loops.