At and beyond the neural limits: visual psychophysics using an adaptive-optics visual stimulator
Lead Research Organisation:
University College London
Department Name: Institute of Ophthalmology
Abstract
The optics of all of our eyes are to some degree imperfect, even those of individuals with "20/20 vision". These imperfection produce small, often imperceptible, aberrations in the images that we see and limits both image quality and visual acuity.
New technology first developed for astronomy, called "adaptive optics" or AO for short, can correct those aberrations and produce nearly perfect, aberration-free images in the eye. Most AO relies on the use of a flexible mirror that can be deformed in a controlled away. The optical aberrations present in the eye are measured and the inverse of those distortions are applied to the mirror, thus correcting the image quality and overcoming the optical aberrations that usually limit image quality in the eye.
AO instruments can be used in two ways in visual science: either to look into the eye or to look out of it. Looking into the eye, instruments that image the retina at the back of the eye can be improved by AO to the extent that individual cells can be resolved. Looking out of the eye, the resolution of a visual image projected onto to retina can be increased by AO to beyond normal visual acuity, producing so called "super-vision". The instrument that we will design and build will be a user-friendly AO visual stimulator with which we can do both of these things. The device will include a digital light projector-normally used to project images in cinemas-that will project super high resolution images directly onto the retina. The primary purpose of the instrument is to investigate the underlying properties of the eye and brain beyond the normal limits imposed by the optics of the eye. Experiments will be conducted to test how well observer are able to see by asking them to make judgments about what they can see.
Once the device is constructed, we will embark on a series of standard measurements of human visual performance. Rather than trying to measure the response to complex visual scenes, we will measure an observer's sensitivity to simple spatial patterns made up of periodic patterns of light and dark. These simple "sinusoidal" patterns are the building blocks from which we can predict the responses to any complex stimuli. To fully characterise the vast space of possible visual scenes, the simple sinusoidal patterns will be systematically varied from coarse to fine. For each pattern, we will determine how much contrast the observer needs to just see the pattern. From these measurements, we derive the spatial contrast sensitivity function (or spatial CSF) that defines the visual performance of the human observer. The importance of the AO system is that we can use it to measure spatial CSFs without the measurements being limited by the optics of the eye. Moreover, we can make those measurements for chromatic stimuli, achromatic stimuli and stimuli detected by single classes of light-sensitive detectors. By using different stimuli, we can investigate different neural pathways in the visual system.
The AO stimulator will also allow us to investigate human vision beyond the usual neural limits of the system. Visual patterns produced by the AO stimulator that are too fine to be seen directly (and are therefore "invisible") can be seen indirectly due to interactions between pairs of patterns or due to changes in apparent intensity when a pattern is turned on and off. Using these patterns we will be able to investigate the underlying properties of the neurons in the visual system. Again, we can make measurements using chromatic stimuli, achromatic stimuli and stimuli detected by single classes of light-sensitive detectors. The indirect detection of these "invisible" patterns allows us to probe the inner working of the pathways answer specific questions about how the retina works.
New technology first developed for astronomy, called "adaptive optics" or AO for short, can correct those aberrations and produce nearly perfect, aberration-free images in the eye. Most AO relies on the use of a flexible mirror that can be deformed in a controlled away. The optical aberrations present in the eye are measured and the inverse of those distortions are applied to the mirror, thus correcting the image quality and overcoming the optical aberrations that usually limit image quality in the eye.
AO instruments can be used in two ways in visual science: either to look into the eye or to look out of it. Looking into the eye, instruments that image the retina at the back of the eye can be improved by AO to the extent that individual cells can be resolved. Looking out of the eye, the resolution of a visual image projected onto to retina can be increased by AO to beyond normal visual acuity, producing so called "super-vision". The instrument that we will design and build will be a user-friendly AO visual stimulator with which we can do both of these things. The device will include a digital light projector-normally used to project images in cinemas-that will project super high resolution images directly onto the retina. The primary purpose of the instrument is to investigate the underlying properties of the eye and brain beyond the normal limits imposed by the optics of the eye. Experiments will be conducted to test how well observer are able to see by asking them to make judgments about what they can see.
Once the device is constructed, we will embark on a series of standard measurements of human visual performance. Rather than trying to measure the response to complex visual scenes, we will measure an observer's sensitivity to simple spatial patterns made up of periodic patterns of light and dark. These simple "sinusoidal" patterns are the building blocks from which we can predict the responses to any complex stimuli. To fully characterise the vast space of possible visual scenes, the simple sinusoidal patterns will be systematically varied from coarse to fine. For each pattern, we will determine how much contrast the observer needs to just see the pattern. From these measurements, we derive the spatial contrast sensitivity function (or spatial CSF) that defines the visual performance of the human observer. The importance of the AO system is that we can use it to measure spatial CSFs without the measurements being limited by the optics of the eye. Moreover, we can make those measurements for chromatic stimuli, achromatic stimuli and stimuli detected by single classes of light-sensitive detectors. By using different stimuli, we can investigate different neural pathways in the visual system.
The AO stimulator will also allow us to investigate human vision beyond the usual neural limits of the system. Visual patterns produced by the AO stimulator that are too fine to be seen directly (and are therefore "invisible") can be seen indirectly due to interactions between pairs of patterns or due to changes in apparent intensity when a pattern is turned on and off. Using these patterns we will be able to investigate the underlying properties of the neurons in the visual system. Again, we can make measurements using chromatic stimuli, achromatic stimuli and stimuli detected by single classes of light-sensitive detectors. The indirect detection of these "invisible" patterns allows us to probe the inner working of the pathways answer specific questions about how the retina works.
Technical Summary
We propose to design and build a user-friendly, adaptive-optics (AO) visual stimulator with which to simultaneously image the retina at high resolution and to present high-resolution visual stimuli. The optical system will include a 3-channel projector capable of running at 120 Hz with a resolution of 1920 x 1080 pixels and with 12-bits of resolution per channel. This will allow us to generate images with high temporal and spatial resolution, high dynamic range and wide colour gamut over an extended intensity range. The components of the system and the technology are proven, so that its development has low technical risk.
Once constructed, we will carry out a series of fundamental measurements of human visual performance. We will determine the human spatial contrast sensitivity functions (CSFs) for detecting achromatic, chromatic, and L-, M- and S-cone-isolating stimuli in the absence of the limitations normally imposed by the optics of the eye. The use of an AO stimulator has significant advantages over the previous measurements obtained using laser interferometric gratings. Steady or drifting equiluminant, equichromatic or cone-isolating gratings or Gabor patches can be presented without the problems of combining spatially-unstable interferometric gratings. In addition, spatial and/or temporal noise can be added to the stimuli to better isolate different channels or processes Measurements will be made over a range of luminances to investigate the effects of adaptation on spatial processing.
As well as going beyond the optical limit, we will investigate retinal processing beyond the usual neural limit. By measuring the distortion products generated by perceptually unresolvable gratings or pairs of gratings, and by applying the linear-nonlinear-linear "sandwich" model, we can separately characterize the spatial filters before and after the site(s) of nonlinear distortion. Initially, gratings that isolate L-, M- or S-cone responses will be used.
Once constructed, we will carry out a series of fundamental measurements of human visual performance. We will determine the human spatial contrast sensitivity functions (CSFs) for detecting achromatic, chromatic, and L-, M- and S-cone-isolating stimuli in the absence of the limitations normally imposed by the optics of the eye. The use of an AO stimulator has significant advantages over the previous measurements obtained using laser interferometric gratings. Steady or drifting equiluminant, equichromatic or cone-isolating gratings or Gabor patches can be presented without the problems of combining spatially-unstable interferometric gratings. In addition, spatial and/or temporal noise can be added to the stimuli to better isolate different channels or processes Measurements will be made over a range of luminances to investigate the effects of adaptation on spatial processing.
As well as going beyond the optical limit, we will investigate retinal processing beyond the usual neural limit. By measuring the distortion products generated by perceptually unresolvable gratings or pairs of gratings, and by applying the linear-nonlinear-linear "sandwich" model, we can separately characterize the spatial filters before and after the site(s) of nonlinear distortion. Initially, gratings that isolate L-, M- or S-cone responses will be used.
Planned Impact
The primary goal of this research project is the elucidation of the fundamental spatial properties of the human visual system. Thus, the initial beneficiaries are likely to be other scientists studying the visual system. They will include visual psychophysicists, sensory physiologists, electrophysiologists and cognitive neuroscientists working on visual processing in the human and primate visual system. Other beneficiaries will include scientists and clinicians comparing normal and abnormal spatial vision.
The spatial contrast sensitivity functions (CSFs) measured with and without adaptive optical (AO) correction will enable vision scientists to predict how well human observers can see other arbitrarily complex visual stimuli both with respect to the image entering the eye and with respect to the image on the retina. Knowing the limits to the performance of the visual system is vitally important, and will allow more precise modelling of visual and neural function. The spatial CSF data will be made available in publications and also on-line at our web resource at http://www.cvrl.org
Clinical vision scientists, optometrists and ophthalmologists will also benefit. The publication of normative spatial CSF data will enable them to assess the extent of the clinical deficits in comparable measurements made in patients, thus aiding in the interpretation and understanding of the disease process and in the design of visual aids that will help patients to improve their quality of life.
While we are using the device mainly to project high resolution images onto the retina, the device can also be used to take high resolution pictures of the retina. This is important not only for basic vision science, since we will able to visualize photoreceptors and link the photoreceptor array to our measurements, but it will be important in the longer term for clinical measurements and diagnosis. In combination, high resolution projection and imaging will allow stimuli targeted to specific areas of the retina that might be of interest or damaged. Early diagnosis of retinal disease is vital and can potentially greatly improve the quality of life of patients. Indeed, a future possibility, given the need and given additional clinical funding, is that we could replicate our instrument at Moorfields Eye Hospital.
The data will also help in the derivation of physiologically-relevant models of human vision as applied in the fields of artificial intelligence or robotics. Knowing the limits of the physiological system will help to define the sensory capabilities required of such systems.
The AO vision system we are building is flexible, so that it can be used for other purposes. For example, it can be used to simulate the visual efficacy of new designs of inter-ocular lenses (IOLs) before they are used in the eye. Other areas of research within spatial vision are all possibilities.
Given the wide range of interest in vision research across many disciplines it will be important that our results and models are widely disseminated. We will publish our work in open source journals and present the data at international conferences. As part of this project, we also propose to make our data available at the Colour and Vision Research Laboratory (CVRL) website run by the PI at http://www.cvrl.org This resource is well-known and widely used in colour research both by academics and in industry. We propose to develop this resource further to provide general information on vision.
The spatial contrast sensitivity functions (CSFs) measured with and without adaptive optical (AO) correction will enable vision scientists to predict how well human observers can see other arbitrarily complex visual stimuli both with respect to the image entering the eye and with respect to the image on the retina. Knowing the limits to the performance of the visual system is vitally important, and will allow more precise modelling of visual and neural function. The spatial CSF data will be made available in publications and also on-line at our web resource at http://www.cvrl.org
Clinical vision scientists, optometrists and ophthalmologists will also benefit. The publication of normative spatial CSF data will enable them to assess the extent of the clinical deficits in comparable measurements made in patients, thus aiding in the interpretation and understanding of the disease process and in the design of visual aids that will help patients to improve their quality of life.
While we are using the device mainly to project high resolution images onto the retina, the device can also be used to take high resolution pictures of the retina. This is important not only for basic vision science, since we will able to visualize photoreceptors and link the photoreceptor array to our measurements, but it will be important in the longer term for clinical measurements and diagnosis. In combination, high resolution projection and imaging will allow stimuli targeted to specific areas of the retina that might be of interest or damaged. Early diagnosis of retinal disease is vital and can potentially greatly improve the quality of life of patients. Indeed, a future possibility, given the need and given additional clinical funding, is that we could replicate our instrument at Moorfields Eye Hospital.
The data will also help in the derivation of physiologically-relevant models of human vision as applied in the fields of artificial intelligence or robotics. Knowing the limits of the physiological system will help to define the sensory capabilities required of such systems.
The AO vision system we are building is flexible, so that it can be used for other purposes. For example, it can be used to simulate the visual efficacy of new designs of inter-ocular lenses (IOLs) before they are used in the eye. Other areas of research within spatial vision are all possibilities.
Given the wide range of interest in vision research across many disciplines it will be important that our results and models are widely disseminated. We will publish our work in open source journals and present the data at international conferences. As part of this project, we also propose to make our data available at the Colour and Vision Research Laboratory (CVRL) website run by the PI at http://www.cvrl.org This resource is well-known and widely used in colour research both by academics and in industry. We propose to develop this resource further to provide general information on vision.
Organisations
Publications
Conway BR
(2018)
A tour of contemporary color vision research.
in Vision research
Rider AT
(2018)
Harmonics added to a flickering light can upset the balance between ON and OFF pathways to produce illusory colors.
in Proceedings of the National Academy of Sciences of the United States of America
Rider AT
(2019)
Light adaptation controls visual sensitivity by adjusting the speed and gain of the response to light.
in PloS one
Stockman A
(2019)
Cone fundamentals and CIE standards
in Current Opinion in Behavioral Sciences
Stockman A
(2017)
Hue shifts produced by temporal asymmetries in chromatic signals depend on the alignment of the first and second harmonics.
in Journal of vision
Stockman A
(2021)
Clinical vision and molecular loss: Integrating visual psychophysics with molecular genetics reveals key details of normal and abnormal visual processing.
in Progress in retinal and eye research
Stockman A
(2018)
Delayed cone-opponent signals in the luminance pathway.
in Journal of vision
Stockman A
(2018)
Delayed S-cone sensitivity losses following the onset of intense yellow backgrounds linked to the lifetime of a photobleaching product?
in Journal of vision
Stockman A
(2017)
Linear-nonlinear models of the red-green chromatic pathway.
in Journal of vision
Stockman A
(2017)
Hue shifts produced by temporal asymmetries in chromatic signals.
in Journal of vision
Description | We made progress in developing the system for testing, but we were unable to get the system working adequately. |
Exploitation Route | We have a functioning adaptive optics device, which we are continuing to develop. The technology of microdisplays is advancing very fast, so we are hoping to be able to purchase high contrast, brighter microdisplays and continue the project. |
Sectors | Other |
Title | CVRL database |
Description | This web resource provides an annotated database of downloadable standard functions and data sets relevant to colour and vision research and to colour technology, as well as providing information about the research outputs of our group. Updated frequently. |
Type Of Material | Database/Collection of data |
Year Produced | 2006 |
Provided To Others? | Yes |
Impact | Widely used in science and industry, the site started at UC San Diego in 1995 and moved to UCL with the PI in 2001. |
URL | http://www.cvrl.org |
Description | BBC World Service, CrowdScience participant. |
Form Of Engagement Activity | A broadcast e.g. TV/radio/film/podcast (other than news/press) |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Public/other audiences |
Results and Impact | CrowdScience participant as an expert on colour vision. |
Year(s) Of Engagement Activity | 2018 |
Description | Bloomsbury 2020 art festival workshop group |
Form Of Engagement Activity | A formal working group, expert panel or dialogue |
Part Of Official Scheme? | No |
Geographic Reach | Local |
Primary Audience | Public/other audiences |
Results and Impact | Workshop over 5 separate days with artists, locals and academics to develop art project for the 2020 Bloomsbury Arts festival. |
Year(s) Of Engagement Activity | 2020 |
URL | http://bloomsburyfestival.org.uk/ |
Description | Colour Group January meeting |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Public/other audiences |
Results and Impact | Organized and ran Colour Group (GB) January scientific meeting online because of COVID. |
Year(s) Of Engagement Activity | 2021 |
URL | https://www.colour.org.uk/ |
Description | Design of new Oriel (Moorfields/UCL) eye hospital. |
Form Of Engagement Activity | A formal working group, expert panel or dialogue |
Part Of Official Scheme? | No |
Geographic Reach | Local |
Primary Audience | Professional Practitioners |
Results and Impact | Working groups and day seminars to design new Oriel eye hospital. Role is Chair of the Dry Labs. |
Year(s) Of Engagement Activity | 2020,2021 |
URL | https://oriel-london.org.uk/latest/ |
Description | Invited speaker, 16th International Symposium on the Science and Technology of Lighting, Sheffield |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Industry/Business |
Results and Impact | 100 attendees and the talk sparked discussion about human vision and lighting. |
Year(s) Of Engagement Activity | 2018 |
Description | Invited speaker, OSA Fall Vision Meeting, Reno, US |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | About 300 attendees. Talk stimulated considerable debate. |
Year(s) Of Engagement Activity | 2018 |
Description | Invited speaker, UP.GRADE seminar on Human vision and color pipelines, Camerimage, Bydgoszcz, Poland. |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Keynote speaker on human colour vision at international film festival. |
Year(s) Of Engagement Activity | 2017 |
Description | Invited talk, Department of Psychology, University of Washington |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | Local |
Primary Audience | Professional Practitioners |
Results and Impact | Seminar presentation. Talk stimulated useful and interesting discussion. |
Year(s) Of Engagement Activity | 2018 |
Description | Keynote speaker 4th China Vision Science Conference (CVSC), Chengdu, China. Speaker at the Chinese National Academy in Beijing. |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Other audiences |
Results and Impact | Keynote speaker at Chinese vision conference; and at Chinese National Academy. |
Year(s) Of Engagement Activity | 2019 |
Description | Keynote speaker, 2nd International conference colour in film, British Film Institute and Colour Group GB. |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Presentation on visual illusions to film makers. |
Year(s) Of Engagement Activity | 2017 |
Description | Lecturer, ICVS summer school, Oxford. |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Postgraduate students |
Results and Impact | International school on colour vision held every two years bu the International Colour Vision Society. Educationally important. |
Year(s) Of Engagement Activity | 2018 |
Description | Lecturer, ICVS summer school, Oxford. |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Postgraduate students |
Results and Impact | Participant in one week summer school on colour vision. |
Year(s) Of Engagement Activity | 2016 |
Description | Member of the Oriel group to design an new hospital/ research institute to house Moorfields Eye Hospital and the UCL Institute of Ophthalmology. |
Form Of Engagement Activity | A formal working group, expert panel or dialogue |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Other audiences |
Results and Impact | Part of the design process for a new eye hospital and research institute for Moorfields Eye Hospital and the UCL Institute of Ophthalmology. Chair of the Dry Labs group, |
Year(s) Of Engagement Activity | 2020 |
URL | https://oriel-london.org.uk/latest/ |
Description | Ophthalmology Grand Rounds talk, Department of Ophthalmology and Visual Sciences, University of British Columbia |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Professional Practitioners |
Results and Impact | Clinical vision talk that was broadcast around British Columbia. Significant positive feedback and interest. |
Year(s) Of Engagement Activity | 2018 |
Description | Participation in Bloomsbury 2020 Arts festival and production of online visual illusions. Replayed 2021 |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Public/other audiences |
Results and Impact | Production of three visual illusion films for festival as an online presentation because of COVID |
Year(s) Of Engagement Activity | 2020,2021 |
URL | https://bloomsburyfestival.org.uk/2020vision/ |
Description | Seminar participant Camerimage film festival, Torun, Poland |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Media (as a channel to the public) |
Results and Impact | Seminar on colour and film |
Year(s) Of Engagement Activity | 2019 |
URL | https://camerimage.pl/en/camerimage-2019/informacje-ogolne/ |
Description | Short course instructor, 24th Color and Imaging Conference, San Diego |
Form Of Engagement Activity | A formal working group, expert panel or dialogue |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Industry/Business |
Results and Impact | 2 hour short course on "Cone fundamentals, color matching functions, luminous efficiency and individual differences". |
Year(s) Of Engagement Activity | 2016 |
Description | Short course instructor, 25th Color and Imaging Conference, Lillehammer |
Form Of Engagement Activity | A formal working group, expert panel or dialogue |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Industry/Business |
Results and Impact | 2 hour short course on "Cone fundamentals, color matching functions, luminous efficiency and individual differences" |
Year(s) Of Engagement Activity | 2017 |
Description | Short course instructor, 26th Color and Imaging Conference, Vancouver. |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Class and workshop in colour and colour vision for people in Colour and imaging. Important for forging links with industry. |
Year(s) Of Engagement Activity | 2018 |
Description | Short course instructor, 27th Color and Imaging Conference, Paris. |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Industry/Business |
Results and Impact | 4 hour workshop on the human visual pipeline from retina to brain. |
Year(s) Of Engagement Activity | 2019 |
URL | https://www.imaging.org/site/IST/Conferences/Color_and_Imaging/CIC27__2019_/IST/Conferences/CIC/CIC2... |
Description | UC Davis, Vision Sciences seminar speaker |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | Local |
Primary Audience | Professional Practitioners |
Results and Impact | Local seminar. Produced good discussion. Lab visits there were very useful and interesting. |
Year(s) Of Engagement Activity | 2018 |
Description | UC San Diego, Department of Psychology Colloquium speaker |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | Local |
Primary Audience | Professional Practitioners |
Results and Impact | Colloquium audience at UC San Diego. Sparked questions and discussion, |
Year(s) Of Engagement Activity | 2018 |
Description | Visual Cognition talk, Department of Ophthalmology and Visual Sciences, University of British Columbia |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Professional Practitioners |
Results and Impact | About 60 attendees enjoyed a talk on colour vision that led to useful discussions and feedback. |
Year(s) Of Engagement Activity | 2018 |