Change detection in complex acoustic scenes

The ability to detect changes around us, such as the appearance or disappearance of an object, has far-reaching implications for survival. These processes have received considerable attention in Vision research. In contrast, the processes by which listeners detect the appearance or disappearance of objects in busy acoustic scenes, comprised of many concurrent sources, remain poorly understood. This is surprising because often it is sound that alerts us to important changes in the scene: Hearing is sensitive to a much larger space than the other senses and in many cases we hear change before we see it (for example, somebody entering the room while your back is to the door; Sudden quiet from the kids' playroom indicating they are up to mischief..). Indeed, the auditory system is commonly assumed to play a key role in the brain's change-detection network by serving as an 'early warning device', rapidly directing attention to new events in the scene.
The present project is classified as 'basic' research with the goal of understanding how listeners with normal hearing detect and process change-events (appearance or disappearance of sources) in auditory scenes. The behavioural and functional brain imaging experiments detailed here are designed to systematically explore listeners' change detection behavior, understand the relevant processes and identify their neural underpinnings: How are object appearance and disappearance events detected? What brain mechanisms are involved? Are change events detected automatically by the brain, even when listeners' attentional focus is elsewhere? What makes certain change events fundamentally more salient than others? Under what conditions do listeners perform well, and which situations result in reduced performance? In busy scenes, such as those we often face in the environment, behaviourally relevant scene changes often coincide in time with other events. How resilient are the auditory change detection mechanisms to irrelevant events occurring at the same time as the auditory change? Do auditory and visual perturbing events have the same detrimental effect on performance?
These issues are important from the point of view of understanding perception and how the brain analyses and represents the dynamics of our surrounding environment. Furthermore, understanding what makes certain events pop out and grab attention, while other events go unnoticed is important for designing human-computer interfaces, and other devices intended to help professionals (operating room personnel, air traffic controllers, pilots, etc.) operate effectively in environments where the detection of change is critical. Additionally, since change detection is a major contributor to efficient interaction with the environment, understanding the profile of change detection in normal listeners can provide a measure against which to evaluate hearing impairment as well as the benefit obtained from hearing aids.

The ability to detect changes around us, such as the appearance or disappearance of objects, has far-reaching implications for survival. The auditory system is commonly assumed to play a key role in the brain's change-detection network by serving as an 'early warning device', rapidly directing attention to new events in the scene, however these processes remain poorly understood. The present project is comprised of psychophysics, as well as functional brain imaging (MEG and fMRI) experiments, designed to understand how (normal hearing) listeners detect the appearance or disappearance of sources in busy acoustic scenes. The experiments employ a new stimulus paradigm, based on tightly controlled acoustic scenes designed to model the dynamics of natural 'soundscapes', but devoid of semantic attributes so as to tap only low level (pre-semantic) processes involved in change detection. Using this paradigm I aim to systematically explore listeners' scene-change detection behavior, in order to identify the underlying brain mechanisms and understand what combination of conditions (physical aspects of sound; presence of other concurrent events in the environment; listeners' perceptual state) make such changes easy to detect, and, conversely, in what circumstances listeners fail to detect prominent changes in acoustic scenes. These issues are important from the point of view of understanding perception and how the brain analyses and represents scene dynamics but also for the design of human-computer interfaces and other devices aimed at aiding professionals operate in 'busy' acoustics scenes where the detection of appearing or disappearing sources is critical. Additionally, since change detection is a major contributor to efficient interaction with the environment, understanding the profile of change detection in normal listeners would be useful for evaluating hearing impairment and measuring the benefit obtained from hearing aids.

The present project is classified as 'basic' research with the goal of understanding how normal hearing listeners detect and process change-events (appearance or disappearance of objects) in auditory scenes. In particular, we aim to understand what combination of conditions (physical structure of sound; presence of other concurrent events in the environment; listeners' perceptual state) make such changes easy to detect, and, conversely, in what circumstances listeners fail to detect prominent changes in acoustic scenes. Understanding the normal 'profile' of change detection could have wide ranging implications for health and wealth.
(1) The ability to detect changes around us is fundamental to successful interaction with the environment. The auditory system plays a major role in this process -we often hear important change-events before we see them (for example, somebody entering the room while your back is to the door; Sudden quiet from the kids' playroom indicating they are up to mischief..). Indeed individuals with hearing impairment tend to rank inability to efficiently respond to such changes among the most debilitating aspects of deafness. However, despite its important implications to quality of life, change detection ability is not currently captured by existing hearing-impairment evaluation techniques which are mainly based on simple audiometry and on measuring speech understanding. Mapping out the characteristics of normal listeners' sensitivity to change can provide a measure against which to evaluate hearing impairment. The testing procedures developed during this project could be used in the clinic or in the context of scientific research, to estimate the degree to which change detection performance differs from 'normal'. This might be useful for evaluating hearing impairment and provide a clearer assessment of benefit to quality of life obtained from a hearing aid. This will benefit individuals, health professionals as well as NHS policy makers.

(2) Furthermore, understanding the properties of listeners' sensitivity to change, what makes certain events pop out and grab attention, while other events go unnoticed is important for designing human-computer interfaces, and other devices intended to help professionals (operating room personnel, air traffic controllers, pilots, etc.) operate effectively in environments where the detection of change is critical. In an age when the environments that such professionals must deal with become increasingly complex, and where many applications actively immerse users in artificial acoustic scenes, understanding the brain processes underlying change detection could inform the design of these systems (for example by programming them so that they produce change events that listeners are particularly sensitive to) or lead to new technological developments that improve auditory change detection through advanced audio signal processing, brain-machine interfaces, etc. Such technological developments could benefit individuals, health professionals, government agencies, and the commercial private sector.

(3) Similarly, Art (design of 'sound-scapes') and the computer gaming industry could benefit from the technological advancements described above, as these fields are currently moving to using artificial environments and brain-machine interfaces for the enhancement of user experience.

(4) Analysis tools developed in the course of this project (especially those related to proposed experiment MEG1) with the aim of relating on-going brain activity to the listener's perceptual experience of change, might also be of use to brain-machine applications thus benefiting government agencies, and the commercial private sector.