Geometric Evaluation of Stereoscopic Video

3D films and games have become a popular part of digital entertainment. There are more and more consumer devices that can show 3D, including the new generation of head-mounted displays, which are reawakening the idea of virtual reality. When it works properly, 3D adds a fun and interesting new dimension to the viewing experience. But sometimes there are problems; for example, the scene may look flat, or distorted. Even worse, there may be too much depth, which can eventually cause eye-strain and headaches.

This project is about evaluating the amount of depth that is being shown, so that video producers and game designers can avoid audience discomfort. There are many rules of thumb for setting the right amount of depth, but it would be safer to take a scientific approach. This project will develop the theory of 3D content evaluation, based on the following principles.

In order to see 3D, the images shown to the left and right eye must be slightly different - just as they are when viewing a real scene. The differences between the two images, which are small horizontal offsets, are called binocular disparities. Roughly speaking, the amount of disparity is proportional to the perceived depth. But in order to fully understand the depth/disparity relationship, it is necessary to know something about the human visual system, as well as the 3D display. In particular, we need to know (or guess) where in the scene the viewer is likely to be looking. The project will develop a system that can predict the point of interest, at any given moment. This information can then be used, in conjunction with the binocular disparities, to quantify how much depth is being shown. The video producer or game designer will then be able to modify the content, so that it includes the right amount of 3D.

The project is at the intersection of several research areas, including geometry, signal processing, and visual perception. A prototype software tool will be developed, in conjunction with a UK visual effects company, for the automatic analysis of 3D content. The final system will be tested by asking ordinary people to evaluate a range of 3D videos; if the system can predict the human responses, then it will be useful to video producers and game designers.

The ultimate impact of this project will be to help expand the range of 3D content, including films, games, visualizations, and virtual reality environments. This will be achieved by developing a new model of stereoscopic discomfort, which will be used by video producers and game designers, to ensure that 3D content is safe to view. This will benefit the general public, who will thereby have access to a wider range of digital entertainment. In particular, the tools developed in this project will help to bring 3D content to the new generation of head-mounted displays, including the Oculus Rift and Sony Morpheus.

Software tools, based on the outcomes of this project, will be developed by engineers and producers in the visual effects industry. This sector is very strong in the UK, and makes an important contribution to the national economy (see Case for support). Hence there will be a medium-term economic benefit to these companies, which are well-placed to engage with the project and its outcomes. Impact will be ensured by working directly with a UK visual effects company (see industrial Letter of support).

More generally, the project will have an impact on 3D cinema standards, and on broadcasting research and development. In addition, there will be an impact on specialist applications, such as stereoscopic medical visualization and security scanning. The impact of the project will be transmitted through presentations and publications of the work, tailored for the engineering, biological and industrial audiences. In addition, a software implementation of the new stereo model will be made freely available on the project website, so that other researchers can adapt and extend it.

In the long term, the model developed in this project will contribute to the understanding of human binocular vision. This has clinical implications for the treatment of oculomotor disorders. The pathway to clinical impact would be via extensive testing of the new model. This could be performed, in a future project, in conjunction with psychophysics and ophthalmology researchers. The UK has a very strong vision research community, which makes this a promising strategy for long-term clinical impact.