Learning to see in depth: neural models of binocular stereopsis

When you go to see a 3D movie, you are provided with a pair of glasses to be worn while in the cinema. You then experience a vivid awareness of the three-dimensional shape of the objects and people in the movie, which appear to leave the screen and occupy space within the room. This occurs because the glasses allow the cinema to present two slightly different versions of the movie to your left and right eye.

The purpose of this proposal is to determine how your brain is able to interpret these differences between what is seen by your two eyes. We know that this is achieved by neurons in the brain that respond to these differences. We will establish how it is able to interpret these, to provide you with a clear appreciation of the three-dimensional shape of objects.

This research is important in providing a good theoretical understanding of binocular vision, so that we may develop successful therapies for conditions such as amblyopia and strabismus ("lazy eye" and "squint") in which binocular depth perception may be impaired or absent.

It is also important in enabling designers of movies and virtual reality systems to make the results as "real" as possible. This research will help us to understand the binocular image differences that our brains respond to, and how it uses these to determine three-dimensional shape. This can then be used to directly inform the design of virtual reality systems.
Finally, it is important in allowing engineers to develop robots and vehicles that can "see"; what we learn about how this is achieved by our brains is very valuable in developing artificial, computer vision.

Binocular vision provides important information allowing us to see in depth. This information is encoded by binocular neurons in the primary visual cortex. We know, however, that depth perception does not arise in any simple manner from the activity in these disparity-tuned neurons. Rather, the perception of depth involves complex exitatory and inhibitory interactions between these neurons, and activity in extrastriate visual areas.

This project will develop models of this processing, based on Independent Components Analysis and the spectral properties of binocular images. We will create a novel set of binocular images with ground-truth data with which to develop and test these models. Our objectives in developing these models are twofold. First, they will provide a theoretical framework with which to understand binocular visual processing in the brain, beyond the initial encoding stage. Second, they will be used to develop novel, biologically-inspired algorithms for binocular stereopsis.

The research has significant potential for impact within and beyond academia:

Academia

Vision Science: The results will be of significance to those studying binocular vision from behavioural and physiological perspectives.

Computer Vision: The development of artificial vision systems has great potential to benefit from our understanding of how our (highly successful) biological visual system operates.

Optometry: A thorough understanding of binocular vision provides the necessary science underpinning the development of therapeutic interventions in cases in which this is impaired.

Business/Industry
The development of biologically inspired stereo algorithms will benefit technicians working in computer vision. Basing our algorithms on state-of-the-art knowledge of neural processing of binocular information will provide new ways of thinking to this field.

The General Public

The recent resurgence of 3D movies means that there is significant public interest in binocular depth perception. There is therefore a demand from the general public to understand the science of binocular depth perception, and an opportunity to use this as an engaging way to present neuroscience.

Pathways to Impact

Hibbard, together with colleagues at other Scottish unviersities, is currently developing the Scottish Vision Group website. This will provide information, and a forum for discussion for a variety of audiences. The first is the general public. We will provide accessible summaries of the state of knowledge of vision, in the context of (i) fundamental science (ii) understanding of disease and sensory impairment and (iii) technology. The second audience is industry and the third-sector; by bringing together the expertise available throughout Scotland in one place, we will make is easy to establish contacts with companies and charities would could benefit from out work. This will allow us to develop our current links (e.g. with the Fife Society for the Blind, and NCR). The third audience is each other, and academics more generally; the website hosts blogs and discussion groups to facilitate networking across the community.
A prototype of the website has been built, and is hosted on servers at St Andrews; the site is scheduled to be made publically available at the end of 2012.

Academic work will be presented at a variety of conferences in neuroscience and machine vision, to ensure impact in both of these fields. Interactive presentations to the general public will be made at the annual science fair hosted by the University of St Andrews.

People

The researcher employed on the project will develop considerable skills that are of value beyond the immediate project. This includes the application of advanced programming skills to a difficult problem in biology, and the analytical, writing and presentation skills required to present the results of the research at conferences, in journal papers, and to the general public.