Perception of colour gradients in real and computer-simulated scenes: effects on depth
Lead Research Organisation:
University of St Andrews
Department Name: Psychology
Abstract
Can you tell the difference between real filmed footage of an event, and a computer-rendered counterpart? Despite tremendous progress in animation and graphics, the answer is most likely yes. We still have a long way to go in generating high quality realistic rendered worlds, that have a wide variety of applications, from gaming, through medical and industrial simulators, to architect-designed walk-throughs that give us a feel for how a new building could look. Improving the naturalness and realism of such virtual environments is a key challenge for those involved in computer graphics and rendering, particularly when there is a demand for interactive, real-time applications: we want to walk around in that simulated new building, not just view static photograph-like scenes. One of the reasons that our progress is slow, is that the extraordinary visual capabilities of most humans, though apparently effortless, hide a complex web of visual processing that is not yet fully understood. If we do not yet understand what enhances realism for the human visual system, it is not surprising that progress is slow in developing technology to improve the realism of simulations. The aim of this work will be to elucidate some of the basic perceptual processes that underlie how subtle changes in colour and lightness enhance the realism of our perception of a three-dimensional scene. This human behavioural research underpins the development of graphics and rendering technologies that will deliver enhanced realism for virtual environments.One of the reasons why this problem is so hard is that the real world contains very complex patterns of light and colour that somehow translate into our perceptions of whether something is green, light, dark, near or far away. For example, my coffee cup on the table in front of me contains white, bright specular highlights that contribute to it looking glossy. There are also reflections of the stripy table mat on which it sits, yet I know these to be reflections, not part of the pattern on the mug. There is an attached shadow, cast by light from the window, and I know that is not part of my mug. And there are changes in lightness and colour across the surface, yet I see the mug as being a single colour, made from a single material. In this project we will study how humans distinguish between depth, light source, and material properties. For most real scenes, this is a very difficult computational problem because specific local patches that appear, for example, darker than those around them, can have the value they do for a variety of reasons. For example, the side of your grey filing cabinet may have a very different lightness than the front, because it slopes away from you in depth, and is at a different angle with the light source, compared with the front of the cabinet (and it could actually be a different colour). How does the human visual system achieve a coherent perception of a solid, non-deforming world, despite changes in view-point and lighting? This is a huge question in both human and computer vision research, with direct implications for computer graphics and virtual environment development, but one that is still poorly understood. We will start by exploring very simple visual scenes containing isolated objects, and study how colour and luminance information can influence depth perception. This data will be used to create models that predict when the luminance/colour information will be most useful. In other experiments we will use real objects and realistically computer-rendered scenes that preserve the relationship between objects and light source that would occur in the real world. Our work will give us immediate information about basic visual mechanisms of depth and scene perception that directly informs the fields of computer graphics and image processing, giving guidelines for when realistic luminance and colour gradients are required for a rendered scene to look realistic.
People |
ORCID iD |
Julie Harris (Principal Investigator) |
Publications
Clery S
(2013)
Interactions between luminance and color signals: effects on shape.
in Journal of vision
Harding G
(2012)
Learning to use illumination gradients as an unambiguous cue to three dimensional shape.
in PloS one
Harris JM
(2014)
Volume perception: Disparity extraction and depth representation in complex three-dimensional environments.
in Journal of vision
Harris JM
(2012)
Perception of relative depth interval: systematic biases in perceived depth.
in Quarterly journal of experimental psychology (2006)
Lovell P
(2011)
How Do Reliability and Timing Influence Cue-Combinations for Shading and Stereo-Disparity?
in i-Perception
Lovell PG
(2012)
Optimal integration of shading and binocular disparity for depth perception.
in Journal of vision
Zeiner KM
(2014)
Perceptual integration across natural monocular regions.
in Journal of vision
Description | This project set out to explore interactions between different visual cues, for the perception of depth and shape. In this project we have studied how humans distinguish between depth, light source, and material properties, and how varying these various parameters impacts on depth and shape perception. In one strand of the project, we studied how colour and luminance information can influence depth perception. We found that realistic colour/luminance gradients can be used to perceive shape (Lovell, Bloj & Harris, 2012), and that although the visual information is often ambiguous, observers can quickly learn the relationship between location, lighting and shape, so that the cue becomes reliable (Harding, Harris & Bloj, 2012). We explored the effects of colour variation on shape-from-shading information. Contrary to previous findings, we discovered that colour does not always enhance perceived depth, as would be expected if the visual system exploited information about the statistics of the natural environment (Clery, Bloj, Harris, 2013, Journal of Vision). Instead, the effects of colour on shape from shading are likely to be due to process in very early vision, where the separate colour and luminance process channels first interact.In the other main strand of the project, we used optimal integration models to demonstrate that shading information can be used as a reliable cue to depth, when combined with binocular disparity, and that a classic model, the Maximum Likelihood Estimator, provides an excellent description of how the brain combines the visual information (Lovell, Bloj, Harris, 2012, Journal of Vision. 12, 1, p. 1-18. doi: 10.1167/12.1.1). Related work has also shown how depth complexity within a display can alter depth perception (Harris, 2014, Zeiner, Spitschan & Harris, 2014). We have also, for the first time, explored how these two visual cues are combined over time. Our research has demonstrated that shape from shading can be a 'quick and dirty' cue to shape at very short viewing durations (where disparity is not useful). At longer durations, binocular disparity becomes the more reliable source of visual information. |
Exploitation Route | To make new progress in visualisation techniques, it is becoming increasingly important to account for human perception and to progress algorithmic research based on a firm empirical and theoretical basis. Our work involved revealing some of the basic perceptual mechanisms required for visual information to be combined for 3D perception. The results have implications for the development of 3D displays and for the development of animated 3D content. The work is also useful for informing those interested in appearance in architecture, museum displays, video game design and other applications requiring an understanding of the brain computations that underlie vision. With this in mind, Harris co-organised a cross-disciplinary conference for vision scientists, technologists and engineers: Predicting Perception: 3rd International Conference on Appearance, 2012. |
Sectors | Aerospace, Defence and Marine,Agriculture, Food and Drink,Construction,Creative Economy,Digital/Communication/Information Technologies (including Software),Education,Healthcare,Culture, Heritage, Museums and Collections,Transport |
Description | The work is useful for informing those interested in appearance in architecture, museum displays, video game design and other applications requiring an understanding of the brain computations that underlie vision. With this in mind, Harris co-organised a cross-disciplinary conference for vision scientists, technologists and engineers: Predicting Perception: 3rd International Conference on Appearance, 2012. |
First Year Of Impact | 2012 |
Sector | Digital/Communication/Information Technologies (including Software),Education,Healthcare |
Impact Types | Economic |
Description | BBSRC responsive mode |
Amount | £307,582 (GBP) |
Funding ID | BB/M001660/1 |
Organisation | Biotechnology and Biological Sciences Research Council (BBSRC) |
Sector | Public |
Country | United Kingdom |
Start | 01/2015 |
End | 01/2018 |
Description | EPSRC St. Andrews DTG PhD funding |
Amount | £90,000 (GBP) |
Organisation | Engineering and Physical Sciences Research Council (EPSRC) |
Sector | Public |
Country | United Kingdom |
Start | 09/2014 |
End | 09/2017 |
Description | Leverhulme Project Grant |
Amount | £200,344 (GBP) |
Organisation | The Leverhulme Trust |
Sector | Charity/Non Profit |
Country | United Kingdom |
Start | 03/2018 |
End | 03/2021 |
Description | ViiHM network: Visual image interpretation in humans and machines |
Organisation | University of Birmingham |
Department | School of Psychology Birmingham |
Country | United Kingdom |
Sector | Academic/University |
PI Contribution | EPSRC funded network, I am a member. |
Collaborator Contribution | A network funded by EPSRC -- Schofield at Birmingham is the PI |
Impact | None yet |
Start Year | 2014 |
Description | Harris Sutton Trust Lectures |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Schools |
Results and Impact | The talk was on vision, part of a Summer School for disadvantaged teenagers, organised by the Sutton Trust charity. Participants were surprised that vision and perception are part of the psychology and neuroscience curricula but were interested and engaged in the topics. |
Year(s) Of Engagement Activity | 2007,2009,2011,2015 |
Description | Harris lab vision exhibit, Fife Science Festival: St. Andrews Open Day |
Form Of Engagement Activity | Participation in an open day or visit at my research institution |
Part Of Official Scheme? | No |
Geographic Reach | Local |
Primary Audience | Public/other audiences |
Results and Impact | Demos and hands-on lab activities from the Harris lab including 3D vision and motion perception, vision and driving, visual camouflage. The activities were popular with all age-groups, from small children, to parents, and older adults. Several people commented that they didn't know vision was linked to neuroscience and psychology, as well as optometry and physics. |
Year(s) Of Engagement Activity | 2010,2011,2012,2015 |
Description | Harris visit to IMAX 2010 |
Form Of Engagement Activity | A formal working group, expert panel or dialogue |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Industry/Business |
Results and Impact | Visit to IMAX (Canada) to discuss possible collaboration and consultancy on 3D film content. |
Year(s) Of Engagement Activity | 2010 |
Description | Harris visit to Technicolor 2011 |
Form Of Engagement Activity | A formal working group, expert panel or dialogue |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Industry/Business |
Results and Impact | Visit to Technicolor, Rennes, France, to discuss possibility for collaboration or consultation on 3D displays. |
Year(s) Of Engagement Activity | 2011 |
Description | Harris: Predicting Perceptions Conference 2012 |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Industry/Business |
Results and Impact | Harris co-organised a conference, Predicting Perceptions: the third international conference on appearance. Edinburgh, April, 2012. the aim was to bring together academics, industrialists, healthcare professionals and others interested in the topic of visual appearance. |
Year(s) Of Engagement Activity | 2012 |
Description | Quiz a whiz interviews, Julie Harris, with Royal Society of Edinburgh |
Form Of Engagement Activity | Engagement focused website, blog or social media channel |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Schools |
Results and Impact | As part of the Royal Society of Edinburgh's Schools program, they deliver interviews with academics, based on questions from school children on the topic of interest. these are broadcast on their own web channel and on their youtube channel. |
Year(s) Of Engagement Activity | 2016,2017 |
URL | https://www.rse.org.uk/schools/students/watch/ |