The early human visual system laid bare by novel techniques and models: psychophysical dissection using complex flicker

Lead Research Organisation: University College London
Department Name: Institute of Ophthalmology

Abstract

The retina is made up of an array of a large number of light-sensitive cells: three types of cones responsible for daytime vision, and a rod responsible for night-time vision. Light that falls on the retina activates these cells, producing electrical responses. These responses are then combined in various ways within different pathways that encode colour and/or brightness and how they change over time and/or space. How these pathways process visual information and how this information is recombined in the brain to produce our perceptions of colour and brightness is not fully understood. Much of the specialization of the visual pathways occurs in the retina itself and cannot be directly examined in humans. Indeed, one of the most important functions of the retina is to process and reduce the amount of information in the visual image before sending it to the brain.

An important feature of all the different pathways is the presence of ON and OFF channels, which carry information about visual attributes that are greater than and smaller than, respectively, the prevailing mean level. The visual signals are split into ON and OFF signals at the first junction between the light-sensitive cells and the subsequent neural components that send the signals to the brain. The visual attributes carried by the ON and OFF channels can be luminance; i.e. brighter or dimmer, or dimensions of colour; i.e., redder or greener, and bluer or yellower. The splitting into ON and OFF channels is often understood to be an efficient and powerful way to process signals that are larger or smaller than the mean, given that neurons in the visual pathway have a resting level (which should be low to save energy) and can only increase their response above this level by a factor of a few hundreds. Whether this is the reason we have ON and OFF pathways is an open question, but the focus of our study is what effects these pathways have on vision, and how these effects can be used to examine the visual system in more detail. We have found that the bifurcation of signals into ON and OFF channels has important implications for vision beyond improving the efficiency of processing and encoding the visual image. And, remarkable, we have found that we can manipulate the signals in the ON and OFF pathways using lights that flicker in complex temporal patterns, and that these manipulations can have profound effects on our perception of colour. Using complex flicker, and a new model that we have developed, we can break down the visual process into its various constituent stages and study each one separately, thereby linking perception to the underlying physiological processes. We can also compare between pathways to see how our percept of red- or greenness differs from our percept of brightness or blue- or yellowness. In this work, we aim to measure and model these kinds of effects and provide new insights into the inner workings of the visual system, leading to a general model of visual processing.

Technical Summary

The main aim of the proposed research is to investigate the inner workings of the visual system using a novel psychophysical technique. The visual signal is split at the first synapse into ON bipolars that signal increments and OFF bipolars that signal decrements. This half-wave rectification is generally considered an efficient way of encoding and processing increments and decrements while maintaining high contrast sensitivity. However, the bifurcation is highly nonlinear and introduces signal components in each pathway, so called distortion products, that were not present in the original stimulus. The temporal properties of vision, however, are often explored using flicker sensitivity and Fourier analysis, which necessarily disregards the effect of any nonlinearities in the system, and considers the whole system as a linear one. Assuming linearity, the response of the visual system to any arbitrary temporal waveform can be constructed from its responses to sinusoids.
We have found that by adding harmonically related signals to sinusoidal flicker, without changing its mean, we can take advantage of the early half-wave rectifiers, and manipulate perception in a red-green colour discrimination task. While the results are complex, they can be explained by a simple model incorporating the half-wave rectification and a static nonlinearity, with linear processes before and between them. The effects vary with frequency allowing us to investigate the linear, frequency-dependent processes separately from the nonlinear, static processes in order to build a more complete model of the visual pathway. We propose to further measure and model the red-green pathway, as well as applying the same techniques to the luminance, blue-yellow and rod pathways. Both spatial and intensity variations will be introduced into our experiments. The models we develop, based on the underlying physiology of the retina, will provide new insights into the links between behaviour and physiology.

Planned Impact

The primary goal of this research project is the elucidation of fundamental properties of pathways in the human and primate visual system. Consequently, the initial beneficiaries of this research will be other scientists studying the visual system. These will include not just visual psychophysicists like the PI and Co-I, but also sensory physiologists, electrophysiologists and cognitive neuroscientists working on visual processing in the human and primate visual system. The initial impact will be, we expect, the provision of models of early visual processing in different pathways that identify the visually significant signals that should also be evident using other objective measurements. Many other scientists, who are less directly involved in vision research, including some geneticists, anatomists working on the early stages of the visual system, and brain-imaging specialists will also potentially benefit from this research. At the more applied level, this work may also benefit engineers interested in the ergonomics of man-machine interfaces. Such interfaces and displays include videos, films, mobile phones and other dynamic presentation devices.

Longer term benefits will include clinical applications of the research. Once we have defined, dissected and modelled the characteristics of different types of visual processing in the ON and OFF pathways in normal observers, and have established the technique of separately stimulating ON and OFF pathways, we can make comparable measurements in patients with specific clinical visual deficits, some of which, such as types of Congenital Stationary Night blindness, specifically affect ON pathways. As we have found with other similar paradigms, this can be a very fruitful way of understanding the nature of the clinical deficit. We already have several ongoing collaborations with our sister institution Moorfields Eye Hospital.

Given the wide range of interest in vision research across many disciplines it will be important that our results and models are widely disseminated. We will publish our work in open source journals and present the data at international conferences. As part of this project, we also propose to make our data available at the Colour and Vision Research Laboratory (CVRL) website run by the PI at http://www.cvrl.org. This resource is well-known and widely used in colour research both by academics and in industry. We propose to develop this resource further to provide general information on vision. Modern applications, such as Wolfram's Computable Document Format and Matlab's Live Scripts and app builder tools, offer great possibilities in creating interactive and visually appealing demonstrations of complex ideas and phenomenon, as well as demonstrations of visual illusions based on our research. We will use the programs to add interactivity to the CVRL website in order to engage more scientists and non-scientists with our work.
 
Description The aims of the project were to examine how lights that flicker with compound waveforms (sums of related sinusoids, which we refer to as harmonic flicker) produce changes in the mean hue or brightness. Our earlier work (Rider et al. 2018, PNAS) showed that L- or M-cone isolating harmonic flicker can produce changes in mean redness and greenness and we developed a model to explain the data, based on ON/OFF pathways and early and late linear filters. Our new findings (paper in preparation) show that achromatic (black and white) harmonic flicker produces changes in the mean brightness of the flickering field. The data are consistent with the existing model with parameter changes to account for the overall faster processing of luminance as well as higher levels of lateral inhibition. We also noted that the L- or M-cone isolating flicker of our earlier work produced noticeable brightness changes, consistent with luminance pathways (L+M) being modulated by these stimuli in addition to chromatic pathways (L-M). Surprisingly, the pattern of brightness changes was not correlated with the pattern of red-green hue changes. We have also found that S-cone isolating harmonic flicker produces mean changes in a purple-lime green axis (the hues associated with S-cone isolated modulations). We have found that the data are consistent with the earlier model but with changes in some of the parameters, consistent with the well-known sluggishness of S-cone pathways. Finally, in adapting the stimuli from L- or M-cone isolating flicker to achromatic, we found that observers were unable to see brightness changes in a mean grey field when the surround was black, but could easily see the change when the background was made the same grey as the mean flicker. We checked to see if this adjustment made a difference to the hue appearance of the cone isolating flicker and found that, remarkably, the hue changes were very different from those with a black background for frequencies below about 8 Hz but very similar at faster frequencies. We still need some more confirmatory data from a small number of observers before these later studies are ready for publication.
Exploitation Route Stimulation of further research into how colour and luminance signals are processed in the human visual system.
Sectors Education

Healthcare

Other

 
Title CVRL database 
Description This web resource provides an annotated database of downloadable standard functions and data sets relevant to colour and vision research and to colour technology, as well as providing information about the research outputs of our group. Updated frequently. 
Type Of Material Database/Collection of data 
Year Produced 2006 
Provided To Others? Yes  
Impact Widely used in science and industry, the site started at UC San Diego in 1995 and moved to UCL with the PI in 2001. 
URL http://www.cvrl.org
 
Description BBC World Service, CrowdScience participant. 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact CrowdScience participant as an expert on colour vision.
Year(s) Of Engagement Activity 2018
 
Description Bloomsbury 2020 art festival workshop group 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact Workshop over 5 separate days with artists, locals and academics to develop art project for the 2020 Bloomsbury Arts festival.
Year(s) Of Engagement Activity 2020
URL http://bloomsburyfestival.org.uk/
 
Description Colour Group January meeting 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Organized and ran Colour Group (GB) January scientific meeting online because of COVID.
Year(s) Of Engagement Activity 2021
URL https://www.colour.org.uk/
 
Description Design of new Oriel (Moorfields/UCL) eye hospital. 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Professional Practitioners
Results and Impact Working groups and day seminars to design new Oriel eye hospital. Role is Chair of the Dry Labs.
Year(s) Of Engagement Activity 2020,2021
URL https://oriel-london.org.uk/latest/
 
Description Invited speaker, OSA Fall Vision Meeting, Reno, US 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact About 300 attendees. Talk stimulated considerable debate.
Year(s) Of Engagement Activity 2018
 
Description Invited talk, Department of Psychology, University of Washington 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Professional Practitioners
Results and Impact Seminar presentation. Talk stimulated useful and interesting discussion.
Year(s) Of Engagement Activity 2018
 
Description Keynote speaker 4th China Vision Science Conference (CVSC), Chengdu, China. Speaker at the Chinese National Academy in Beijing. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact Keynote speaker at Chinese vision conference; and at Chinese National Academy.
Year(s) Of Engagement Activity 2019
 
Description Member of the Oriel group to design an new hospital/ research institute to house Moorfields Eye Hospital and the UCL Institute of Ophthalmology. 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Other audiences
Results and Impact Part of the design process for a new eye hospital and research institute for Moorfields Eye Hospital and the UCL Institute of Ophthalmology. Chair of the Dry Labs group,
Year(s) Of Engagement Activity 2020
URL https://oriel-london.org.uk/latest/
 
Description Ophthalmology Grand Rounds talk, Department of Ophthalmology and Visual Sciences, University of British Columbia 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Clinical vision talk that was broadcast around British Columbia. Significant positive feedback and interest.
Year(s) Of Engagement Activity 2018
 
Description Participation in Bloomsbury 2020 Arts festival and production of online visual illusions. Replayed 2021 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact Production of three visual illusion films for festival as an online presentation because of COVID
Year(s) Of Engagement Activity 2020,2021
URL https://bloomsburyfestival.org.uk/2020vision/
 
Description Seminar participant Camerimage film festival, Torun, Poland 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Seminar on colour and film
Year(s) Of Engagement Activity 2019
URL https://camerimage.pl/en/camerimage-2019/informacje-ogolne/
 
Description Short course instructor, 26th Color and Imaging Conference, Vancouver. 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Class and workshop in colour and colour vision for people in Colour and imaging. Important for forging links with industry.
Year(s) Of Engagement Activity 2018
 
Description Short course instructor, 27th Color and Imaging Conference, Paris. 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact 4 hour workshop on the human visual pipeline from retina to brain.
Year(s) Of Engagement Activity 2019
URL https://www.imaging.org/site/IST/Conferences/Color_and_Imaging/CIC27__2019_/IST/Conferences/CIC/CIC2...
 
Description UC Davis, Vision Sciences seminar speaker 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Professional Practitioners
Results and Impact Local seminar. Produced good discussion. Lab visits there were very useful and interesting.
Year(s) Of Engagement Activity 2018
 
Description UC San Diego, Department of Psychology Colloquium speaker 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Professional Practitioners
Results and Impact Colloquium audience at UC San Diego. Sparked questions and discussion,
Year(s) Of Engagement Activity 2018
 
Description Visual Cognition talk, Department of Ophthalmology and Visual Sciences, University of British Columbia 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact About 60 attendees enjoyed a talk on colour vision that led to useful discussions and feedback.
Year(s) Of Engagement Activity 2018