Functional role of second-order processing in binocular vision

Lead Research Organisation: University of Stirling
Department Name: Psychology


The human brain uses small differences between the images reaching our two eyes to perceive the three-dimensional shape of the world around us. In order to detect these differences, known as binocular disparities, the brain must find points in one eye's image that match to points in the other eye's image. However, for any single point in an image, the brain is often forced to choose between multiple matches. The problem of finding the correct match from amongst these alternatives is known as the correspondence problem. This problem can be simplified by making assumptions about the typical shape of objects in the world, and by finding matches between different kinds of basic tokens. For example, the number of alternative solutions to the correspondence problem will be far greater if the brain uses single points of light as a basic token for matching, compared to a case where a more complex token, such as a shape or texture, is used. Furthermore, these complex tokens can be based on different kinds of information. The research proposed here examines how matching tokens based on different forms of information can be used by the brain to solve the correspondence problem. Specifically, we shall examine how the brain may solve the correspondence problem using tokens derived from mechanisms sensitive to changes in light and dark (i.e. changes in luminance), and mechanisms sensitive to changes in texture. We shall develop computer simulations of the processes used by the brain to solve the correspondence problem and measure disparity. These simulations will show how the use of different basic information for matching (i.e. changes in luminance and changes in texture) can change the nature of the correspondence problem. We shall discover whether the combined use of texture- and luminance-based matching tokens can help to reduce noise in disparity measurement and whether the use of texture-based matching can reduce the number of available solutions to the correspondence problem. Following this, we shall examine whether the brain actually makes use of the combined information available from texture and luminance. By presenting human participants with images containing disparities defined by both texture and luminance, we shall establish whether the human brain actually uses these different types of information to reduce noise, or improve its ability to solve the correspondence problem. In addition to examining whether using luminance and texture information to measure disparity helps the brain to reduce noise and simplify the correspondence problem, we shall also examine whether sensitivity to these different types of image information can help the brain to detect discontinuities in depth. Depth discontinuities arise when depth changes sharply across a small area, such as when an observer's view of one object is partially obscured by another object in front of it. The processing of texture-based disparities may help in the detection of depth discontinuities since different objects often differ in texture. We shall establish whether information of this kind may actually be useful in the detection of depth discontinuities, and whether human observers actually use this information.

Technical Summary

Small differences between the images falling onto the retinas of our two eyes are detected by the human visual system and used to recover information about the three-dimensional structure of the environment. If these differences, known as binocular disparities, are to be properly encoded, the visual system must first find matching points between the two images. Through a combination of computational analysis and psychophysical experimentation, the proposed research will investigate the mechanisms used by the human visual system to find matching points and accurately encode the structure of the environment. Specifically, the proposed research will examine the visual system's use of second-order information for the encoding of binocular disparity. By analysing the second-order content of binocular natural images, we will establish ways in which such information may prove helpful in solving the correspondence problem. A series of psychophysical experiments shall establish whether potentially helpful second-order information is used by the visual system to solve the correspondence problem. Further analyses of natural image data will establish whether second-order image content can provide useful information about the spatial locations of discontinuities in depth. Psychophysical experiments will further establish whether any potential usefulness of second-order information is actually exploited by the visual system for the purpose of discontinuity detection.


10 25 50
Description This research has examined the processes that allow humans to perceive depth using the information available with two eyes. This information, known as binocular disparity, can be used to perceive depth even when three-dimensional (3D) structure cannot be seen with only one eye. This is known as cyclopean perception. To perceive cyclopean structure the brain must overcome two problems. First, it must measure the disparities in the image. Second, it must group together those disparity measurements into descriptions of 3D properties. Our research addresses both these issues.

To examine the measurement problem, we have conducted an analysis of the disparity information available in binocular natural images. Model disparity sensitive cells, similar to those found in the macaque brain, were used to process such images. Model cell responses were then analysed in order to quantify the complexity of the measurement problem in natural scenes. Our results suggest that the measurement problem can be reduced through a combination of multiple types of disparity sensitive cells, including cells that are sensitive to variations in image contrast.

In a further examination of disparity measurement, we have looked at how human observers perform in tasks requiring the perception of cyclopean structure, in the presence of measurement noise. Measurement noise was added by randomly replacing parts of the image, by randomly moving image parts away from the cyclopean surface, by reducing contrast (the difference between light and dark), and by varying the orientation of lines used to depict cyclopean surfaces. Our results suggest that different forms of noise affect different processing levels and that combinations of noise types severely disrupt disparity measurement.

In examining the problem of cyclopean grouping, we have looked at how the addition of image elements with random locations in depth affects the visual system's ability to recover cyclopean structure. The observed reduction in performance is consistent with a model that groups pairs of image features over a local area, and provides further processing stages with a description of the relations between those pairs. Such a model will be of use in the production of general descriptions of cyclopean structure, that allow for comparisons of performance to be made across different tasks and stimuli. Importantly, our results cannot be accounted for by models that make no attempt to group together multiple disparity signals over space.

In further examinations of the problem of cyclopean grouping, we have looked at the visual system's understanding of clouds of random dots in depth. This investigation has revealed that that we have little sensitivity to the shape of such clouds and rely only on estimates of their extent in depth. We have also looked at the role of disparity in grouping image elements into contours. Our results show that similarity of disparity plays a role in contour detection, but that its importance is outweighed by other factors such as contour linearity and element orientation.
Exploitation Route Our findings on the importance of grouping processes in the perception of cyclopean form demonstrate the need for models of disparity processing that go far beyond simple disparity measurement. Currently, our model may only be applied to a limited subset of (random-dot) stimuli, and may only be used to measure a limited set of cyclopean properties. Further extensions of this model will allow for a far fuller description of cyclopean form. Such a model may be applied to coupled camera-computer systems to allow for 3D image analysis, addressing problems beyond the production of 3D maps. Thus the models we are developing may be of use in robot navigation and 3D imaging applications.

Finally, our findings on the perception of stereoscopic volumes (clouds in depth), provide significant limitations on the use of stereoscopic displays for 3D data visualisation. If observers possess only a limited description of the shape of a cloud in depth, then stereoscopic visualisation technologies must provide additional signals to aid users. These results also hold similar implications for designers of 3D virtual environments.
The research conducted on this grant offers several potential strands for exploitation. First, the results of our image analysis demonstrate the importance of applying multiple disparity sensor types in order to solve the measurement problem. We have highlighted several potential sensor types, and show that the measurement problem is distinct in each. Our image analysis code may be made available to the scientific community in order to aid further development of models of disparity measurement.

Second, our models of cyclopean grouping offer significant opportunities for exploitation. Currently, however, they offer only limited use. Further development of these models would allow for greater applicability. Collaboration with the computer vision community would offer the possibility to fully develop the concepts used in our grouping model.
Sectors Digital/Communication/Information Technologies (including Software)