Steering visual attention in natural scenes: efficient image rendition by actively guiding fixation
Lead Research Organisation:
University of Manchester
Department Name: Electrical and Electronic Engineering
Abstract
What features of an object or region of a scene attract attention? Research over the last few decades has identified several basic features, including movement, colour, brightness contrast, and orientation. This information is important in understanding the mechanisms and strategies that seeing organisms use in their interactions with the environment. Curiously, much less research has been carried out on what properties of objects or regions do not attract attention. In addition to improving our understanding of biological vision, this knowledge could be important in applications such as image and video coding where there is limited channel capacity. If a region of an image receives little or no attention by a viewer, then less channel bandwidth need be allocated to its representation. If, moreover, it is possible to actively influence the probability of a particular location attracting attention by applying image-processing methods, then still less bandwidth might be required. The general aim of the present work is to understand better what regions of a natural scene are represented incompletely or not at all under natural viewing and to determine how this information might be used to steer attention towards other areas of an image to achieve optimum image coding for an electronic display system or more effective advertising or signage. Two practical deliverables are a pilot scheme for reducing video bandwidth and a real-time image display that incorporates image processing to guide fixation to important regions.
Organisations
People |
ORCID iD |
| David Foster (Principal Investigator) | |
| John Oakley (Co-Investigator) |
Publications
Amano K
(2012)
Visual search in natural scenes explained by local color properties.
in Journal of the Optical Society of America. A, Optics, image science, and vision
Amano K
(2014)
Influence of local scene color on fixation position in visual search.
in Journal of the Optical Society of America. A, Optics, image science, and vision
Amano K.
(2010)
Tracking categorical surface colour across illuminant changes in natural scenes
in 5th European Conference on Colour in Graphics, Imaging, and Vision and 12th International Symposium on Multispectral Colour Science 2010, CGIV 2010/MCS'10
Feng G
(2012)
Predicting frequency of metamerism in natural scenes by entropy of colors.
in Journal of the Optical Society of America. A, Optics, image science, and vision
Foster DH
(2019)
Hyperspectral imaging in color vision research: tutorial.
in Journal of the Optical Society of America. A, Optics, image science, and vision
Foster DH
(2011)
Color constancy.
in Vision research
Foster DH
(2018)
The Verriest Lecture: Color vision in an uncertain world.
in Journal of the Optical Society of America. A, Optics, image science, and vision
John P Oakley
(2011)
Colour and gaze as predictors of target detection in natural scenes
Marín-Franch I
(2010)
Number of perceptually distinct surface colors in natural scenes.
in Journal of vision
| Description | Looking for an object in the world around us is a common part of everyday experience. Success or otherwise depends on many factors, including the structure of the environment or scene being searched, the nature of the object being sought, and the task itself. Much of what is known about the physical factors determining visual search performance has come from laboratory experiments with simple abstract displays of geometric elements defining a target in a background of distractors. Less is known about visual search in natural scenes, with most work concentrating on the separate question of eye movements and visual saliency. The general aim of the present work was to understand better what properties of natural scenes are important in attracting or steering attention and in facilitating target detection. Application areas include optimum image coding for electronic display systems and more effective advertising and signage. In a series of laboratory experiments, observers were given a search task with images of natural scenes taken from a set of hyperspectral data, rendered on a colour monitor with a target embedded at a random location in the scene. The target was constructed from a small, shaded, grey sphere, matched in mean luminance to its local surround. Observers' gaze was simultaneously monitored with an infra-red video eye-tracker. The main experimental outcomes were as follows. (1) About 60% of the variation in observers' target-detection performance over each scene was accounted for by the variation in local scene colour properties (luminance, chroma, and hue). A similar level of variation was accounted for by the pattern of observers' eye fixations. These levels are similar to the maximum level reported elsewhere for the effects of scene structure on eye movements in free viewing of black-and-white video images. Thus, despite a general assumption that image factors such as local contrast and edges dominate visual search and gaze, the present results suggest that local colour information can be as important as gaze in determining target-detection performance in natural scenes. (2) In viewing natural scenes, however, observers may exploit the familiarity, meaning, and global organisation of such scenes to facilitate target detection. To provide a control on the effects of global organisation, images were electronically cut into quarters, randomly rearranged, and then reassembled. Target detection was little affected with these scrambled images, reinforcing the importance of local scene properties. (3) To see whether observers' fixations could be guided over the scene, the local contrast in regions of the image was selectively increased or decreased in such a way as to be largely unnoticed by the observer. Although it was difficult to shift attention away from regions by decreasing contrast, dynamically increasing local contrast did attract gaze and improve target detection. Critically, this image manipulation was generally less detectable than the target itself, suggesting that it gaze can indeed be subtly manipulated to enhance performance. (4) In the course of analysing observers' gaze behaviour, a computational method was developed to make more objective the classification of fixations, for which there is no standard method. General distributional properties of eye movements were used to classify fixations without the parametric assumptions or expert judgment required by other methods. The method was verified against independent classifications by experts and by computer simulation. Software for the proposed nonparametric method is to be made freely available to the community. |
| Exploitation Route | Machine assisted search and target detection, automatic analysis of eye gaze in natural scenes, improving signage, navigation |
| Sectors | Aerospace, Defence and Marine,Digital/Communication/Information Technologies (including Software) |
| URL | http://personalpages.manchester.ac.uk/staff/d.h.foster/Fixation_Classification/Classifying_Fixations.html |
| Title | Table of Color Constancy Indices and Experimental Methods |
| Description | Levels of color constancy from some experimental studies. Eighty average constancy indices CI and Brunswik ratios BR are tabulated against experimental method, stimulus configuration, illuminants, judgment by subject, experimental apparatus, illuminant change, cues other than those defined by the experimental method, constraints on stimulus variation, the number of subjects N, and source of data. Values of CI and BR are averages over subjects and conditions, with some entries estimated from published figures. |
| Type Of Material | Database/Collection of data |
| Year Produced | 2011 |
| Provided To Others? | Yes |
| Impact | According to Google Analytics, 167 pageviews per month |
| URL | http://personalpages.manchester.ac.uk/staff/d.h.foster/Table_colour_constancy_indices.html |