Rank based spectral estimation

The colours, or RGB pixels, recorded by a digital camera are the result of the interaction of the prevailing light in the scene striking and being reflected by objects and the characteristics of the camera itself. The complexity is such that different cameras see differently and no cameras see the world exactly as we do. You will have noticed this when looking at photos where sometimes the colours don't look right or the pictures captured by one camera look 'better' than another. Moreover, sometimes we see colours change dramatically. We have all probably observed that white clothes can look bluish under ultra violet light (say in a night club). But, in fact the colours we see change subtly, all the time, as we move from one light to another (which is why it is always a good idea to check the colour of your clothes outside the shop). Here, even small changes can lead to poor customer satisfaction or, potentially, in a medical imaging application the wrong diagnosis.

Good pictures, by which we might mean accurate 'colour measurement' are possible if we know the spectral colour characteristics of a camera and/or the spectrum of light in a scene. While we can, in principle, measure these quantities the measurement is not easy to do so and is expensive (not easy as it requires considerable (Physics) lab time and expensive because spectral measurement devices cost many thousands of pounds). When measurement is not feasible, there do in fact exist methods for estimating (say) the spectrum of light in a scene. Yet, these methods only tend work if the camera is accurately calibrated first (a sort of chicken and the egg situation). Our 'Rank Based Spectral Estimation' Project aims to make it much easier to calibrate a camera or measure the illuminant in situ (and as such also make it easier to measure reflectance too)

So, how does our method work. Well suppose we gave you 50 grey tiles all of which appeared to have a different brightness. It would be an easy task for you to rank them from darkest to brightest. But, now suppose we change the colour of the light. Depending on the spectral shape of the grey reflectances, the ranking order can change (sometimes considerably). No problem, it is a simple matter to reorder the tiles. Remarkably, for specially chosen reflectances, the rank order will strongly correlate with the spectral shape of the light. Thus a simple ranking experiment gives us a strong clue to the colour of the light. (And, if we knew the colour of the light we could, for example predict whether the colour of our clothes might change when we go outdoors.)

The Rank Based Spectral Estimation project aims to take this simple ranking idea and provide simple, and accurate, estimation tools for deriving the spectral shape of the prevailing light, the spectral characteristics of a camera and the spectral reflectances of surfaces. At the heart of our method is a specially designed reflectance target containing many reflectances (whose design is part of the proposed research). Ranking these reflectances will allow us to accurately estimate the light spectrum and the spectral attributes of a camera. Accurate spectral estimates are required in many applications from photography, through, visual inspection to forensic imaging and telepresence (e.g. remote diagnosis).

Remarkably, we believe the methods we develop will also prove useful in understanding how we see. Indeed, it is very likely that you see the world a little differently than I do. Yet estimating an individual's spectral response is notoriously difficult. To the extent it can be done at all, it requires many hours of (tedious) detailed visual experiments. Through ranking it will be possible to uncover an observers spectral response (technically called 'colour matching curves') quickly and simply. We simply ask the observer to carry out a simple ranking of the kind mentioned above.

Rank-based Spectral Estimation is a new paradigm for estimating the spectrum of the light, the spectral characteristics of the camera and for making spectral reflectance estimation easier. RBSE will be developed directly with our project partners who have each identified an estimation problem they are interested in solving where the traditional (expensive and time consuming) measurement approaches are difficult to use. With Apple we will develop and evaluate RBSE for the spectral estimation of camera spectral sensitivities paying special attention to the problem of the 'batch to batch' variation' in the spectral response of cameras. Unilever has a long-standing interest in modelling the appearance of physical objects which, hitherto, has relied on lab-based measurements. Through RBSE we aim to help Unilever make in situ appearance measurements. The interest of Datacolor is color measurement. They are the leading manufacturer of colour measurement devices. Through our collaboration we will be able to assess how well RBSE can approximate physical measurement. Finally, with Buhler-Sortex we will investigate the color tolerances needed for stable accept/reject decisions in industrial inspection. RBSE will be used to help calibrate Sortex machines.

Of course through our extensive dissemination plans, we will also publish our developed algorithms and data. So, any industry that has a need for spectral estimation will be able to prototype our technology (including, in medical and forensic imaging). More generally, we will extend http://www.cvrl.org where Prof Stockman provides access to human colour vision datasets as well as making a dedicated project website. These resources will include tutorial articles to ensure that the general public can engage with our research.

Both PIs on this grant have an extensive track record of communicating their research to standards bodies. This includes CIE technical committees TC 8-02 and TC8-07 in Image Technology and TC 1-72 on measurement of appearance. Prof Stockman has also worked closely on the definition of a new standard on cone response functions (TC 1-36: Fundamental Chromaticity Diagram with Physiologically Significant Axes). Also, Teresa Goodman chairs TC 2-65: Methods of Characterizing Spectrophotometers. Prof Finlayson has also worked with ISO TC 42 (Photography) for which the Society of Imaging Science and Technology (for which Finlayson serves as a Vice President) has an oversight and administration role.

The project also provides unrivaled training opportunities for the PDRA (Dr Vazquez) and the UEA funded research student funded alongside this grant. It is our plan that both will be trained in optical radiation measurement and, under the supervision of Teresa Goodman at NPL. And both will will be trained in psychophysical experimentation under the primary supervision of Prof. Stockman at UCL. Dr Vazquez will also spend time working directly in the labs of Buhler Sortex and Unilever (see their letter of support). The whole project team will benefit from the large and varied integration with industry. The whole team will have the opportunity to see their research transferred and then exploited in an industrial context.

In 2013 the Quadrennial meeting of the International Colour Society meets in Newcastle. This is a forum for scientists, industrialists, practionners and the lay public to meet and discuss colour. We will make a coordinated series of presentations as well as a purpose built exhibit at this meeting. In 2008, NPL (leading from research with partners including Unliever) had a very successful exhibit at the Royal Society Summer Exhibition on the 'Physics of Perception'. We will submit a proposal to exhibit for summer 2014. More generally the team has a history of tutorial presentations to organisations such as 'The Colour Group of Great Britain' and 'The Institute of Electrical Engineers' and meetings of the RPS Imaging Science Group.