The timecourse of utilising high-level information in scene perception

Lead Research Organisation: University of Dundee
Department Name: Psychology

Abstract

Abstracts are not currently available in GtR for all funded research. This is normally because the abstract was not required at the time of proposal submission, but may be because it included sensitive information such as personal details.
 
Description Experiment 1 showed that expectations about both target appearance and placement are utilised to guide eye movements from the beginning of scene inspection, orienting the direction of the first saccade. In both search initiation and subsequent scanning, the availability of a visual template was particularly useful when the spatial context of the scene was misleading. The availability of a reliable scene context facilitated search mainly when the template was verbal. Therefore, the visual system can utilise target template guidance and context guidance flexibly from the beginning of search, depending upon the amount and the quality of the available information supplied by these high-level sources.

Experiment 2 explored in more detail the effects of spatial inconsistency in scene search, teasing apart any effect of attentional prioritisation due to inconsistency (context/local information conflict) from the disrupting impact of unreliable expectations. We replicated the findings of Experiment 1 but also found that spatial violations depending on distractors may diminish search efficiency attracting early attention when these objects are the biggest in the target expected region and the target template is not precise (verbal). This experiment thus showed, for the first time in the visual search domain, that objects are initially selected covertly considering their relative size and that subsequent overt selection is made considering object-scene associations processed in extrafoveal vision.

The project has provided key contribution to our understanding of visual search:
1. Both expectations about target appearance and expectations about target placement in scenes are used to guide search from the first eye movement.
2. If precise (visual) target information is known prior to search, object placement within the scene is unimportant. If only imprecise (verbal) information is available, search is disrupted if the target is not where expected.
3. Matching with template features in working memory and processing of object-scene associations occur to a considerable extent already in extrafoveal vision.

The experiments conducted have acted as catalysts for further work in our lab. Three follow-up studies have been run:
1.The role of object colour, comparing search following coloured or greyscale templates in coloured or grayscale scenes. 24 participants tested.
2.The importance of assimilation time after the search template, comparing 100ms/900ms ISIs between template and scene following verbal or visual information. 48 participants tested.
3.Functional specialisation of the cerebral hemispheres in early search guidance. When targets in scenes are lateralised the hemispheric specialisation for language (LH) or visuospatial information (RH) may result in different search initiation after verbal and visual templates. 104 participants tested across 4 experiments.

We have created the first publicly available online resource containing not only eye-movement data and experimental scenes but also ratings of scene complexity and the subjective salience, semantic importance and plausibility of placement for target objects (from 10 individuals). This will be a useful resource for researchers studying plausibility of object placement in scenes. Experiment 1 and 2 data and images are available at http://www.activevisionlab.org/.
Exploitation Route Our work provides new insights into the role of consistency in eye guidance and visual search. This will be of importance to those working in vision research and psychology and will extend our current scientific knowledge. We are the first to demonstrate an effect of inconsistency that is separable from effects of template guidance and spatial expectation guidance in scenes. Our method in Experiment 2 provides a means of separating these different influences on eye guidance that has not previously been described or used. Thus the method and images will be of benefit to the field.

Understanding how the eyes are guided to regions in scenes where objects are expected and how inconsistently placed objects capture (or do not capture) attention in scenes has potential wider benefit outside academia. Specifically it may inform understanding of situations in which search for unexpected or misplaced objects is essentially, such as security tasks like CCTV surveillance and baggage screening. In particular if the precision of target knowledge influences the extent to which the eyes are distracted by inconsistency or by spatial expectations about placement, this can be used to refine search protocols in security settings (e.g., providing an image of a search target will help avoid distractions compared to providing a verbal description).
Sectors Security and Diplomacy,Transport
URL http://activevisionlab.org/research/timecourse/
 
Title Eye movement dataset - Experiment 2 
Description Dataset of eye movement records for participants viewing images in experiment 2 of the ESRC awarded grant. Raw eye movement records are made available publicly. Data provided are those used in Spotorno et al. (2015) JOV 
Type Of Material Database/Collection of data 
Year Produced 2016 
Provided To Others? Yes  
Impact This is the first publicly available dataset of eye movement behaviour when viewing scenes with inconsistently placed critical objects, placed alongside a number of distracter objects in each of two scene regions. These data will be of use to those working on eye movements, scene viewing and scene-object inconsistency. 
URL http://activevisionlab.org/research/timecourse/
 
Title Eye movement dataset - experiment 1 
Description Dataset of eye movement records for participants viewing images in experiment 1 of the ESRC awarded grant. Raw eye movement records are made available publicly. Data provided are those used in Spotorno et al. (2014) JOV 
Type Of Material Database/Collection of data 
Year Produced 2014 
Provided To Others? Yes  
Impact This is the first publicly available dataset of eye movement behaviour when viewing scenes with inconsistently placed objects. These data will be of use to those working on eye movements, scene viewing and scene-object inconsistency. 
URL http://activevisionlab.org/research/timecourse/
 
Title Image database - Experiment 2 
Description Database of images used in Experiment 2. These are photographic scenes with photographic objects inserted. Each scene appears in 2 versions with differing placements of critical objects, along with a number of distracter objects in each of two scene regions. Critical objects are placed plausibly or implausibly in the scene. All scenes and object insertions were tested for a variety of measures by independent raters. 
Type Of Material Database/Collection of data 
Year Produced 2016 
Provided To Others? Yes  
Impact This is the first image dataset of its kind and will be of use to those working on aspects of scene inspection and in particular on the impact of inconsistency of object placement in scenes. 
URL http://activevisionlab.org/research/timecourse/
 
Title Image database - experiment 1 
Description Database of images used in experiment 1. These are photographic scenes with photographic objects inserted. Each scene appears in 4 versions with differing placements of objects. Objects are placed plausibly or implausibly in the scene. All scenes and object insertions were tested for a variety of measures by independent raters. 
Type Of Material Database/Collection of data 
Year Produced 2014 
Provided To Others? Yes  
Impact This is the first image dataset of its kind and will be of use to those working on aspects of scene inspection and in particular on the impact of inconsistency of object placement in scenes. 
URL http://activevisionlab.org/research/timecourse/
 
Description Collaboration with George Malcolm 
Organisation George Washington University
Department Department of Psychology
Country United States 
Sector Academic/University 
PI Contribution We collaborated with George Malcolm throughout the period of award and continue to do so. During the award period we ran the project and were responsible for the day-to-day management of the research. We worked with Dr Malcolm to create the experimental programmes and then collected the data. We developed the statistical approaches for analysing the data. We contributed intellectual input at all stages of the work. We have been the main contributors to the dissemination of the findings of this collaboration in papers and talks.
Collaborator Contribution Dr Malcolm provided intellectual input throughout the period of award and since then in our continued collaboration. For the project, Dr Malcolm contributed heavily to the construction of materials (scenes) for the Experiments and contributed to writing the research papers.
Impact Spotorno, S., Malcolm, G. L. & Tatler, B. W. (2014). How context information and target information guide the eyes from the first epoch of search in real-world scenes. Journal of Vision, 14 (2):7, 1-21. Spotorno, S., Malcolm, G. L. & Tatler, B. W. (2015). Disentangling the effects of spatial inconsistency of targets and distractors when searching in realistic scenes. Journal of Vision, 15 (2): 12, 1-21.
Start Year 2011