Illuminating Colour Constancy: from Physics to Photography

In daily life, we depend on colour images which represent the real world, from photographs of key personal events to pictures of possible purchases. In general, these are poor approximations of the real thing. Our aim is to understand better how we perceive colours in the real world, and how to recreate that perception with images. Central to these aims is colour constancy, a fundamental phenomenon which keeps object colours stable even under large changes in the colour of the illumination - we see an apple as red whether it is under bluish daylight or yellowish tungsten light. Camera sensors, which faithfully record the changing light signals, do not naturally possess colour constancy. But digital cameras are often equipped with special colour balancing modules to cope with changes in lighting, and the photographs they produce may be further processed to remove colour casts. In computer vision, such 'color correction' algorithms are necessary to enable machines to use colour as a reliable cue - for example, in automated grading of manufactured goods such as tiles. Human vision and computer vision are typically studied in isolation from each other: the first aims to understand why colours appear as they do to humans, and the other to make them as useful as possible to machines, regardless of how they appear. These two goals are generally not identical, because neither human nor computer colour constancy is perfect.To bridge colour constancy from humans to machines we will perform an innovative set of experiments. First, we will systematically study illuminant metamerism. Metamerism is what makes all image reproduction work: two stimuli with vastly different colour spectra can induce the same colour percept. The light invoking a white percept on a TV has a highly spiky spectrum compared to the flat spectral reflectance of a piece of white paper in daylight. Yet, illuminants which look the same when shining on white paper can sometimes make other surfaces change appearance. We experience this phenomenon when we buy clothes which look good under the artificial shop lights but less satisfactory when we take them outdoors. We will quantify this effect for real scenes under real lights using a new 'tuneable' spectral illuminator with which we can generate any light spectrum. Our second innovation is to make use of newly available High-Dynamic-Range (HDR) displays. In contradistinction to the real world where the brightest point in the scene may be a 100000 times as bright as the darkest point, most displays struggle to produce a dynamic range of even 1000:1 and printed photographs are at most 100:1. Yet we know that colour perception depends on the overall dynamic range of the scene. The new HDR displays can output contrast ratios of 100000:1 and we will use them to measure constancy in lab conditions but with real world brightnesses. A third challenge that we face in making colour photographs match our perception of the real world is the inaccuracy of colour memory. Typically, when we view a photograph, we do not have the real thing to compare it with, but must recall the original scene from memory. The imperfections of our memory then may taint our judgment. It is well known that our memory colours for familiar objects such as sky, grass, and skin tend to be 'over-saturated' -- grass may be remembered as greener and the sky as bluer than they actually are. Thus, when we test colour correction algorithms by asking people which image they prefer, we might find that they do not prefer the one that most accurately reproduces the original scene, but instead matches their imperfect memory. We will quantify these effects of memory and preference. Finally, our research will, at all stages, consider how measured percepts of colour might be predicted by mathematical models. Ultimately, we will design algorithms to automatically see colours as we do, making for better photographs and more useful vision machines.

Colour images are ubiquitous in daily life, used for professional, commercial, and personal purposes, yet they often fail to reproduce an object or scene as it would appear in 'real life' - in art reproductions, holiday photographs, product depictions in catalogues or websites, newscasts, and more. This failure generally arises because colour images do not successfully incorporate colour constancy. Colour constancy poses an important engineering problem for the design of devices that produce colour images -- in many domains, from digital cameras, television, remote sensing, displays, printing and industrial inspection. Indeed, Edwin Land (founder of Polaroid), argued that colour constancy was the fundamental problem in colour vision. Our research aims to provide a step change in our understanding of the theory and practice of colour constancy, advancing both models of human perception as well as colour correction algorithms for devices. It will, by definition, target a broad range of beneficiaries: both end-users such as professional and amateur photographers, designers, artists, marketing personnel, computer game designers and the public in general; and the commercial users who will ultimately deliver the improvements in colour images, such as camera and display manufacturers and imaging software companies. Our direct partners will benefit most immediately from this research programme but will also ensure its high impact. Imsense Ltd is a spin out company from the University of East Anglia exploiting Prof. Finlayson's work on dynamic range compression (www.imsense.com). The company also have an interest in other aspects of image quality, including colour constancy. Dolby, who make proprietary software packages for high dynamic range (HDR) displays, are interested in understanding how human observers see colour on HDR displays. Our data will help them make better pictures. The CIE (The International Commission on Illumination, http://www.cie.co.at/index_ie.html) are also partners on this project and they have agreed to help us disseminate our work to industry and academics through CIE Division 8 (which focuses on Imaging Technology). Professor David Brainard (U. Pennsylvania) is a world-leading authority on colour constancy who has agreed to collaborate on all aspects of the project. The impact of our research will be significantly enhanced through our links with Prof Brainard, especially for dissemination in the USA. More generally, colour constancy and colour image processing are key areas in computer vision and image processing, as well as in visual psychophysics. We wish to ensure the widest impact to these communities and beyond. As part of the project plan we will deliver both new data and data analysis tools to these communities. A variety of new algorithms will also be developed and these will also be released, as a Matlab toolbox. Another key deliverable of this project is a database of calibrated, high dynamic range, multispectral images. To our knowledge this type of image set does not yet exist and we expect many follow-on works to make use of this data. We are also planning to present our work to scientific meetings that have a broad scientific appeal including the Digital Futures meetings of the Royal Photographic Society (http://www.rps-isg.org/), selected monthly meetings of the Colour Group (www.colour.org.uk), whose audience includes graphic designers, display engineers, photographers, artists, museum curators and others, the International Colour Association congress (AIC 2013), which will be held in Newcastle, and (we hope) at a Royal Society Summer Exhibition meeting (2011/2012). See impact plan for details.