The early human visual system laid bare by novel techniques and models: psychophysical dissection using complex flicker

The retina is made up of an array of a large number of light-sensitive cells: three types of cones responsible for daytime vision, and a rod responsible for night-time vision. Light that falls on the retina activates these cells, producing electrical responses. These responses are then combined in various ways within different pathways that encode colour and/or brightness and how they change over time and/or space. How these pathways process visual information and how this information is recombined in the brain to produce our perceptions of colour and brightness is not fully understood. Much of the specialization of the visual pathways occurs in the retina itself and cannot be directly examined in humans. Indeed, one of the most important functions of the retina is to process and reduce the amount of information in the visual image before sending it to the brain.

An important feature of all the different pathways is the presence of ON and OFF channels, which carry information about visual attributes that are greater than and smaller than, respectively, the prevailing mean level. The visual signals are split into ON and OFF signals at the first junction between the light-sensitive cells and the subsequent neural components that send the signals to the brain. The visual attributes carried by the ON and OFF channels can be luminance; i.e. brighter or dimmer, or dimensions of colour; i.e., redder or greener, and bluer or yellower. The splitting into ON and OFF channels is often understood to be an efficient and powerful way to process signals that are larger or smaller than the mean, given that neurons in the visual pathway have a resting level (which should be low to save energy) and can only increase their response above this level by a factor of a few hundreds. Whether this is the reason we have ON and OFF pathways is an open question, but the focus of our study is what effects these pathways have on vision, and how these effects can be used to examine the visual system in more detail. We have found that the bifurcation of signals into ON and OFF channels has important implications for vision beyond improving the efficiency of processing and encoding the visual image. And, remarkable, we have found that we can manipulate the signals in the ON and OFF pathways using lights that flicker in complex temporal patterns, and that these manipulations can have profound effects on our perception of colour. Using complex flicker, and a new model that we have developed, we can break down the visual process into its various constituent stages and study each one separately, thereby linking perception to the underlying physiological processes. We can also compare between pathways to see how our percept of red- or greenness differs from our percept of brightness or blue- or yellowness. In this work, we aim to measure and model these kinds of effects and provide new insights into the inner workings of the visual system, leading to a general model of visual processing.

The main aim of the proposed research is to investigate the inner workings of the visual system using a novel psychophysical technique. The visual signal is split at the first synapse into ON bipolars that signal increments and OFF bipolars that signal decrements. This half-wave rectification is generally considered an efficient way of encoding and processing increments and decrements while maintaining high contrast sensitivity. However, the bifurcation is highly nonlinear and introduces signal components in each pathway, so called distortion products, that were not present in the original stimulus. The temporal properties of vision, however, are often explored using flicker sensitivity and Fourier analysis, which necessarily disregards the effect of any nonlinearities in the system, and considers the whole system as a linear one. Assuming linearity, the response of the visual system to any arbitrary temporal waveform can be constructed from its responses to sinusoids.
We have found that by adding harmonically related signals to sinusoidal flicker, without changing its mean, we can take advantage of the early half-wave rectifiers, and manipulate perception in a red-green colour discrimination task. While the results are complex, they can be explained by a simple model incorporating the half-wave rectification and a static nonlinearity, with linear processes before and between them. The effects vary with frequency allowing us to investigate the linear, frequency-dependent processes separately from the nonlinear, static processes in order to build a more complete model of the visual pathway. We propose to further measure and model the red-green pathway, as well as applying the same techniques to the luminance, blue-yellow and rod pathways. Both spatial and intensity variations will be introduced into our experiments. The models we develop, based on the underlying physiology of the retina, will provide new insights into the links between behaviour and physiology.

The primary goal of this research project is the elucidation of fundamental properties of pathways in the human and primate visual system. Consequently, the initial beneficiaries of this research will be other scientists studying the visual system. These will include not just visual psychophysicists like the PI and Co-I, but also sensory physiologists, electrophysiologists and cognitive neuroscientists working on visual processing in the human and primate visual system. The initial impact will be, we expect, the provision of models of early visual processing in different pathways that identify the visually significant signals that should also be evident using other objective measurements. Many other scientists, who are less directly involved in vision research, including some geneticists, anatomists working on the early stages of the visual system, and brain-imaging specialists will also potentially benefit from this research. At the more applied level, this work may also benefit engineers interested in the ergonomics of man-machine interfaces. Such interfaces and displays include videos, films, mobile phones and other dynamic presentation devices.

Longer term benefits will include clinical applications of the research. Once we have defined, dissected and modelled the characteristics of different types of visual processing in the ON and OFF pathways in normal observers, and have established the technique of separately stimulating ON and OFF pathways, we can make comparable measurements in patients with specific clinical visual deficits, some of which, such as types of Congenital Stationary Night blindness, specifically affect ON pathways. As we have found with other similar paradigms, this can be a very fruitful way of understanding the nature of the clinical deficit. We already have several ongoing collaborations with our sister institution Moorfields Eye Hospital.

Given the wide range of interest in vision research across many disciplines it will be important that our results and models are widely disseminated. We will publish our work in open source journals and present the data at international conferences. As part of this project, we also propose to make our data available at the Colour and Vision Research Laboratory (CVRL) website run by the PI at http://www.cvrl.org. This resource is well-known and widely used in colour research both by academics and in industry. We propose to develop this resource further to provide general information on vision. Modern applications, such as Wolfram's Computable Document Format and Matlab's Live Scripts and app builder tools, offer great possibilities in creating interactive and visually appealing demonstrations of complex ideas and phenomenon, as well as demonstrations of visual illusions based on our research. We will use the programs to add interactivity to the CVRL website in order to engage more scientists and non-scientists with our work.