Estimating the intrinsic characteristics of real images to aid analysis

Humans find seeing things effortless and this hides the fact that making sense of the visual world is a very difficult problem. Vision is difficult because each image we see could have been made by an infinite number of object and lighting combinations. Think of the simplest image property - grey level. The grey level of each pixel in an image is determined by the amount of light falling onto each object and the amount of light that is reflected back from each object. Dark objects have lower grey levels than light ones but even light objects have low grey levels when in shadow. We cannot tell whether we are looking at a dark object in bright light or a light object in shadow just by measuring grey levels. Even worse, when grey levels are different in different parts of an image we cannot tell if this difference is due to there being two objects or a change in the amount of light. Despite this problem humans are very good at working out the reasons for grey level changes; we CAN tell objects from shadows.One reason why we are so good at working out what's going on in images maybe that we use other properties such as colour and pattern to tell us what the grey levels mean. This idea has led to the concept of 'intrinsic images'. An intrinsic image is an image that describes one property of the scene. So instead of having a single image that mixes up shadows and object reflectances we might produce two intrinsic images one each for shadows and reflectance. Scientists have already succeeded in producing intrinsic images like these by using colour changes to work out what the grey levels mean. But, there is more than one type of shadow and more than one type of reflection. We want to improve on the existing methods by producing four intrinsic images instead of two. Our first intrinsic image will contain the type of gentle shading that is found on undulating surfaces. Our second intrinsic image will contain the hard shadows that are produced when an object blocks the light. Our third intrinsic image will describe the reflectance of matte objects and our forth image the reflections from shiny objects. To separate out these four images we will need to use additional information beyond colour. We think that surface patterns (e.g. wood grain) will provide the necessary information.Extracting four intrinsic images will be very helpful to those engineers who try to make computers understand what's going on in an image. To take just one example, humans seem to be very good at is estimating the shape of undulations on a surface from the way that it is shaded. We are so good at this that we do it automatically and the people who write computer software can trick us into thinking that their 'buttons' stand out from the screen just by adding a some highlights and shading to the edges. There are many computer programs that try to interpret shape-from-shading. While many of these programs work well they tend to assume that all changes in grey level are due to shading which is in tern due to surface undulations. We know that this assumption is not true in real pictures and these programs tend to do badly when looking at such images. But if we can produce shading only images from real images then these programs may work better.To test our ideas and decide on the best way achieve our desired results we will collect a large number of photographs of objects whose shape we either already know or can work out. We will calibrate these pictures very carefully and then use them to workout what information is conveyed by colour and pattern that can help us to workout the meaning of each grey level change. We will also test humans to see which cues they might be using. We will make our images available on the Internet so that others can try out their ideas too. We intend to work with a software company who will take the best of our ideas and implement them in a computer program that can automatically design embossed jewellery from photographs.