Perception of colour gradients in real and computer-simulated scenes: effects on depth

Can you tell the difference between real filmed footage of an event, and a computer-rendered counterpart? Despite tremendous progress in animation and graphics, the answer is most likely yes. We still have a long way to go in generating high quality realistic rendered worlds, that have a wide variety of applications, from gaming, through medical and industrial simulators, to architect-designed walk-throughs that give us a feel for how a new building could look. Improving the naturalness and realism of such virtual environments is a key challenge for those involved in computer graphics and rendering, particularly when there is a demand for interactive, real-time applications: we want to walk around in that simulated new building, not just view static photograph-like scenes. One of the reasons that our progress is slow, is that the extraordinary visual capabilities of most humans, though apparently effortless, hide a complex web of visual processing that is not yet fully understood. If we do not yet understand what enhances realism for the human visual system, it is not surprising that progress is slow in developing technology to improve the realism of simulations. The aim of this work will be to elucidate some of the basic perceptual processes that underlie how subtle changes in colour and lightness enhance the realism of our perception of a three-dimensional scene. This human behavioural research underpins the development of graphics and rendering technologies that will deliver enhanced realism for virtual environments.One of the reasons why this problem is so hard is that the real world contains very complex patterns of light and colour that somehow translate into our perceptions of whether something is green, light, dark, near or far away. For example, my coffee cup on the table in front of me contains white, bright specular highlights that contribute to it looking glossy. There are also reflections of the stripy table mat on which it sits, yet I know these to be reflections, not part of the pattern on the mug. There is an attached shadow, cast by light from the window, and I know that is not part of my mug. And there are changes in lightness and colour across the surface, yet I see the mug as being a single colour, made from a single material. In this project we will study how humans distinguish between depth, light source, and material properties. For most real scenes, this is a very difficult computational problem because specific local patches that appear, for example, darker than those around them, can have the value they do for a variety of reasons. For example, the side of your grey filing cabinet may have a very different lightness than the front, because it slopes away from you in depth, and is at a different angle with the light source, compared with the front of the cabinet (and it could actually be a different colour). How does the human visual system achieve a coherent perception of a solid, non-deforming world, despite changes in view-point and lighting? This is a huge question in both human and computer vision research, with direct implications for computer graphics and virtual environment development, but one that is still poorly understood. We will start by exploring very simple visual scenes containing isolated objects, and study how colour and luminance information can influence depth perception. This data will be used to create models that predict when the luminance/colour information will be most useful. In other experiments we will use real objects and realistically computer-rendered scenes that preserve the relationship between objects and light source that would occur in the real world. Our work will give us immediate information about basic visual mechanisms of depth and scene perception that directly informs the fields of computer graphics and image processing, giving guidelines for when realistic luminance and colour gradients are required for a rendered scene to look realistic.