Incorporating vertical disparity into computational models of depth perception

Lead Research Organisation: Newcastle University
Department Name: Sch of Psychology

Abstract

Our two eyes provide us with two slightly different windows on the world. Our brains combine these to give us a unified view of the world, while using the subtle differences between them to impart a rich perception of the world in 3D. Understanding this process will tell us more about perception in general, as well as suggesting how we might in future be able to help people in whom this process has failed, for example in squint. Our brains have developed very precise circuitry for controlling eye movements, in order to ensure that both eyes always look at the same thing and in order to minimise any vertical offsets between the images. When we look straight ahead, the two eye’s images differ only by horizontal offsets, reflecting the horizontal displacement of our eyes. However, when we move our eyes around a visual scene, small vertical disparities are inevitable. Does our brain predict the expected vertical disparities, given the current position of the eyes? Or does it simply allow for some range, treating vertical disparities as noise in the system? The answer will help us understand the sophisticated computations which underlie our effortless perception.

Technical Summary

A fundamental problem of binocular vision is matching up points on the two retinae which are viewing the same object in space. If our eyes were fixed in the head, always pointing forwards to infinity, the images of a given object would always be horizontally offset from one another, and this stereo correspondence problem would be one-dimensional. However, the freedom to move our eyes around a visual scene necessarily introduces vertical disparities, making the problem two-dimensional. This represents a key constraint on oculomotor control: our eyes are precisely yoked together so as to keep vertical disparities to a minimum on the retina. However, it is impossible to remove vertical disparities altogether, and it is clear that the brain detects the residual vertical disparities and uses them, together with extra-retinal information on eye position, to convert horizontal disparity into a perception of depth and surface structure.

In recent years, the response properties of disparity-selective neurons in several cortical areas have been thoroughly documented. Theoretical neuroscientists have begun to examine how populations of such neurons could be used to solve the correspondence problem and support depth perception. However, so far these models have largely ignored vertical disparity and eye movements. The goal of this project is to extend previous models to take account of these, thereby expanding the explanatory power of such models to a much wider range of stimuli. To achieve this, we shall first conduct a series of human psychophysics experiments designed to reveal how vertical disparity affects stereo correspondence. These will examine the scale at which vertical disparity is encoded, and whether it is detected by dedicated populations of neurons or simply deduced from the response of neurons to horizontal disparity. They will reveal whether the brain takes account of the changing patterns of vertical disparity experienced as the eyes move, either by changing the range within which stereo matches can be found at all, or by adjusting the a priori probability assigned to each possible match.

The results of these experiments, together with existing psychophysical and physiological knowledge, will be used to build the first neuronally-realistic stereo correspondence algorithm which handles two-dimensional disparity and allows for eye movements. The inputs to the algorithm will be realistic models of disparity-tuned neurons, while its outputs will be designed to capture human psychophysics. This model will represent one of the most complete attempts to date to link single neurons and perception.

Publications

10 25 50
 
Description Biomedical Vacation Scholarship
Amount £1,440 (GBP)
Organisation Wellcome Trust 
Sector Charity/Non Profit
Country United Kingdom
Start 07/2010 
End 08/2010
 
Description Flexibility and Sustainability Award
Amount £8,000 (GBP)
Organisation NHS England 
Sector Public
Country United Kingdom
Start 04/2010 
End 08/2010
 
Description Flexibility and Sustainability Award
Amount £18,586 (GBP)
Organisation NHS England 
Sector Public
Country United Kingdom
Start 10/2010 
End 09/2012
 
Description Leverhulme Trust Research Leadership Award
Amount £960,528 (GBP)
Funding ID RL-2012-019 
Organisation The Leverhulme Trust 
Sector Academic/University
Country United Kingdom
Start 04/2013 
End 03/2018
 
Description Psychophysical tests for cortical inhibition 
Organisation Newcastle University
Department Institute for Ageing and Health
Country United Kingdom 
Sector Academic/University 
PI Contribution Serrano-Pedraza and Read, together with students in the lab, identified suitable stimuli from the literature, wrote stimulus code, solved technical issues of presentation, tested on control groups and contributed experimental apparatus for clinical tests.
Collaborator Contribution Identified interesting patient group to study, advised on stimulus design, ethics and safety, monitors patients EEG during experiments, contributes clinical expertise.Contributed the original idea by identifying potentially exciting result to examine, provides basic science expertise.
Impact Award for NHS Flexibility and Sustainability funding (£18,586). Poster presentations at International League Against Epilepsy meeting, and at European Conference on Visual Perception meeting. The need to develop and test stimuli in control groups provided experimental projects for two Masters students, and has undercovered interesting results which are expected to result in two publications (one paper currently under review at Journal of Vision, one in preparation). This collaboration is multi-disciplinary, between clinical neurophysiology, in vitro electrophysiology, and human visual psychophysics. After the completion of award G0601566 and the departure of Dr Serrano-Pedraza, the collaboration begun in this way led to a 3-year PhD studentship funded by Epilepsy Action (added in Further Funding) and (as of 2015) two further publications, which I have not added here as I thought they were not directly enough related to the award. However, one can certainly trace a chain back to this award.
Start Year 2010
 
Description Psychophysical tests for cortical inhibition 
Organisation Newcastle upon Tyne Hospitals NHS Foundation Trust
Country United Kingdom 
Sector Public 
PI Contribution Serrano-Pedraza and Read, together with students in the lab, identified suitable stimuli from the literature, wrote stimulus code, solved technical issues of presentation, tested on control groups and contributed experimental apparatus for clinical tests.
Collaborator Contribution Identified interesting patient group to study, advised on stimulus design, ethics and safety, monitors patients EEG during experiments, contributes clinical expertise.Contributed the original idea by identifying potentially exciting result to examine, provides basic science expertise.
Impact Award for NHS Flexibility and Sustainability funding (£18,586). Poster presentations at International League Against Epilepsy meeting, and at European Conference on Visual Perception meeting. The need to develop and test stimuli in control groups provided experimental projects for two Masters students, and has undercovered interesting results which are expected to result in two publications (one paper currently under review at Journal of Vision, one in preparation). This collaboration is multi-disciplinary, between clinical neurophysiology, in vitro electrophysiology, and human visual psychophysics. After the completion of award G0601566 and the departure of Dr Serrano-Pedraza, the collaboration begun in this way led to a 3-year PhD studentship funded by Epilepsy Action (added in Further Funding) and (as of 2015) two further publications, which I have not added here as I thought they were not directly enough related to the award. However, one can certainly trace a chain back to this award.
Start Year 2010
 
Description Suppression in intermittent exotropia 
Organisation Newcastle upon Tyne Hospitals NHS Foundation Trust
Country United Kingdom 
Sector Public 
PI Contribution Read and Serrano-Pedraza, in collaboration with our partners, devised the experiments. Serrano-Pedraza wrote all the code to run the experiments and analyse the data. Read provided the visual images used for stimuli and auditory files used for feedback. Serrano-Pedraza and Read led on writing the papers.
Collaborator Contribution Identified interesting patient group to study; helped design experiments; contacted suitable patients and invited to participate; helped run experiments.
Impact Results were presented at the 2010 meetings of the Vision Sciences Society and the Association for Research in Vision and Ophthalmology in the USA, where in the latter case they won a Novartis prize paying the presenting author's travel expenses. The results are due to be presented at the Clinical Grand Rounds and Clinical Neurosciences Forum in Newcastle shortly. The results have been published in the leading vision journal "Investigative Ophthalmology and Visual Science" (see "Visual suppression in intermittent exotropia during binocular alignment" under Publications section) and we also published a review ("Single vision during ocular deviation in intermittent exotropia" in Publications). We were awarded £8000 of NHS Flexibility and Sustainability Award to pursue this project. The collaboration is multidisciplinary, between basic vision science and clinical ophthalmology. The experience this gave me in binocular vision disorders, and the collaboration established between me (Dr Read) and Drs Serrano-Pedraza and Dr Clarke was a key factor in our current £1M Health Innovation Challenge Fund award, "Accurate and patient-friendly measurement of binocular visual function using a 3D mobile device" (http:/research.ncl.ac.uk/asteroid), added in Further Funding.
Start Year 2008
 
Description  
IP Reference  
Protection Copyrighted (e.g. software)
Year Protection Granted
Licensed Commercial In Confidence
Impact We are about to license our ASTEROID vision test app to a Canadian company for sale. This is why I am now reporting it. It's not a direct output from my original grant, but I think the grant was a key point on my research trajectory and it's unlikely I would have developed this app without getting that grant.
 
Title ASTEROID 
Description This is a set of vision tests on a digital 3D tablet, currently assessing visual acuity and stereoacuity 
Type Of Technology Software 
Year Produced 2016 
Impact Software is currently used by a number of research groups around the world and is soon to be licensed for sale. This is not a direct outcome of the "vertical disparity" grant (I do not acknowledge that grant on papers, for exaqmple), but I doubt I would have produced this outcome if I hadn't got the "vertical disparity" grant if you see what I mean.I assume that by continuing to ask for outcomes 8 years after the grant ended ,you are interested in indirect as well as direct outcomes. 
 
Description Cafe Scientifique 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? Yes
Geographic Reach Local
Primary Audience Schools
Results and Impact 5 sixth-formers attended a talk on neurological disorders.

I got positive feedback.
Year(s) Of Engagement Activity 2007
 
Description Identity Masquerade 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? Yes
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact 3 short films funded by the Wellcome Trust were shown at the Centre for Life, dealing with issues of identity raised by modern neuroscience. I was on hand to attempt to answer any questions arising.

None.
Year(s) Of Engagement Activity 2010
URL http://www.jennyreadresearch.com/misc/public-engagement/
 
Description Nuffield summer placements 
Form Of Engagement Activity Participation in an open day or visit at my research institution
Part Of Official Scheme? Yes
Geographic Reach Local
Primary Audience Schools
Results and Impact Hosted 5 sixth-formers in my lab for 6 weeks doing a research project. 4 of them ended up being authors on scientific papers

Each student wrote a report, presented a poster, and successfully applid for a gold CREST award based on their work. Apparently both have been enthusing to teachers and fellow students. Have been asked to speak at the school of one of them.
Year(s) Of Engagement Activity 2010,2011
URL http://www.jennyreadresearch.com/people/sixth-formers/
 
Description School visits, multiple 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? Yes
Geographic Reach Local
Primary Audience Schools
Results and Impact I have given 5 talks to schools since 2007 as a Science Ambassador and not as part of other schemes (described elsewhere). I don't know how to interpret the instructions above only to report engagement relating to my MRC-funded research. I talked to them about visual neuroscience, but obviously I did not get into the finer points of vertical disparity and binocular geometry when talking to an audience of 7 year olds; I simply show them some cool visual illusions and discuss how your brain achieves vision.

I got positive feedback.
Year(s) Of Engagement Activity 2007,2008,2009,2010
URL http://www.jennyreadresearch.com/misc/public-engagement/
 
Description Science Week talks 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? Yes
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact I gave twotalks to local schools, and delivered a public lecture as part of Newcastle's annual series of Holmes lecture which this year were part of Science Week.

Lots of interest and lots of volunteers signed up for our Institute's Research Volunteer Scheme, which is a database of local people willing to serve as controls in psychology experiments.
Year(s) Of Engagement Activity 2009
URL http://www.jennyreadresearch.com/misc/public-engagement/
 
Description Summer Science Programme in Kyoto 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? Yes
Geographic Reach International
Primary Audience Schools
Results and Impact Gave a talk on face recognition, simultaneously translated into Japanese. I was at the IUPS meeting to talk specifically about the work funded by this grant, but it would not have been accessible to talk about that work to the children, and I was asked to talk about face recognition.

Good feedback.
Year(s) Of Engagement Activity 2009
URL http://www.jennyreadresearch.com/misc/public-engagement/
 
Description mini medical school 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? Yes
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact Gave a talk on visual neuroscience to a public audience largely made up of sixth-formers hoping to read medicine

Lots of interest, was told later that one participant decided to read neuroscience at university as a result, and another ended up doing a Nuffield bursary in my lab.
Year(s) Of Engagement Activity 2009
URL http://www.jennyreadresearch.com/misc/public-engagement/
 
Description neuroscience workshops 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? Yes
Geographic Reach Local
Primary Audience Schools
Results and Impact Groups of school children crafted neurons out of plasticine

Good feedback.
Year(s) Of Engagement Activity 2010
URL http://www.jennyreadresearch.com/
 
Description personal website 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact I added paragraphs on my website describing in accessible terms the papers resulting from this study.

None I can tie directly to this study, but in general the large and informative website I maintain about my research does attract interest and emails both from other researchers and from the lay public.
Year(s) Of Engagement Activity 2009,2010,2011
URL http://www.jennyreadresearch.com/misc/public-engagement/