Multi-modal cue integration for auditory spatial location by normal-hearing and hearing-impaired listeners.

A sharp sense of space is essential to making the world appear natural, real and joined-up. People with normal hearing can do exceptionally well in experimental tasks that measure the sense of space - someone's "spatial hearing". Yet hearing impairment generally worsens spatial hearing, leaving only an impoverished impression of auditory space. Hearing aids hardly ever help spatial hearing. Worse, they can often add further problems. But despite decades of research the science of spatial hearing is still not solved, especially for complex situations with multiple sounds happening simultaneously or in understanding the effects of impairment or aiding. This programme will help answer these puzzles.

There are two overall benefits to listening from having acute spatial hearing. Both take advantage of that fact that in everyday listening sound sources are almost always in distinct physical locations. First, it allows sound sources to be perceived in the locations that they really are. Second, it allows sounds to be separated by virtue of being in different positions. These two together give a sense of realism to the auditory world and help identifying, recognizing or understanding sounds. These reduce the "clutter" of hearing in busy, noisy situations: without any spatial hearing at all, everything would be more likely to be heard together as a jumble.

The situation is complicated by there being multiple cues to the direction and distance of a sound source. One is time: the sound from a source to the left will arrive a fraction of a second later at the right ear compared to the left as it has to travel some 50 cm further around the head. Another is intensity: the sound is less powerful as your head casts an acoustic "shadow". Yet despite these multiple cues we generally hear one location for a sound, not many. Further, what we see can affect where we hear a sound -- a famous example of this is the ventriloquist effect. The problem of generating accurate perceptions of auditory space is, overall, akin to solving a three-dimensional jigsaw of multiple overlapping sounds, time delays, and power differences. Somehow our perceptual systems seamlessly and apparently effortlessly solves the puzzle of putting all the pieces together properly.

In this programme, we will use cutting-edge auditory experiments to answer two key questions. First, how does the auditory system join the multiple cues to location in complex, dynamic, multi-sound, audio-visual listening situations? Second, how does hearing impairment and aided listening affect this? We expect that the insights gained in this programme will help us to understand better how spatial hearing works in real, everyday listening, and will help inform how future hearing aids might be designed to improve spatial hearing

A faithful perceptual representation of the spatial position of sound sources is fundamental to making sense of the acoustic world. It is required to give a real appearance to the auditory scene, and helps considerably in separating sounds and enhancing listening. But hearing impairment worsens spatial perception, reducing or removing these benefits, and hearing aids can actively interfere with it. The perceptual problem of generating the representation is particularly complicated as there are multiple separate cues to location - not only auditory but also visual and vestibular - yet somehow these are combined seamlessly. Quite how is unclear. The overarching purposes of this programme are therefore to determine (1) how this integration of information, across frequency, time, space and audio or visual modality is done and (2) how it is affected by hearing impairment and aided listening.

We will use innovative auditory & audio-visual psychophysical experiments to recreate the real world in the laboratory, emphasizing complex scenes with multiple objects. These will exploit figure-ground separation, so with the same stimuli we can quantify the contribution of spatial cues to both a figure's auditory position and its distinctiveness. We will also exploit front-back indeterminacy to study the effects of vision. Data will be collected using methods such as discrimination of position, subjective reporting of position, and orientation tracking to changes in position.

The research will help deliver our long-term aim of developing a fully detailed scientific account of spatial hearing in everyday, noisy listening: i.e., how someone's perception of a sound source's position is determined from the source's actual position in the world. Such a theory will help identify new phenomena which future hearing devices can exploit or mitigate.

We will seek to continue IHR's excellent track record of disseminating our scientific results and knowledge to wider communities (eg http://www.100yearsofamplifiedmusic.org/). In this programme we plan to deliver two non-academic impact goals: (1) increase public awareness of hearing and hearing impairment in general and spatial-hearing in particular, and (2) interact with hearing-device developers so that our science can lead to improved future devices that benefit spatial hearing instead of interfering with it.

To do these we will engage three audiences:

A. Health Engagement: We expect that the insights found in this programme will be of interest both to many of those with hearing impairment and to many health-professionals in audiology and ENT. Though the results will not lead to an immediate improvement in hearing aids (but see Industry Engagement below for downstream effects) we expect that they will be of background benefit through increasing their understanding of hearing and the emphasis on its problems (goal #1).

B. Industry Engagement: Our results will be of great interest to the hearing-device industry, as both improving the spatialness of sounds delivered by hearing aids and developing visually-aware hearing aids are current foci of industrial research. We confidently hope that this programme will feed into improvements in future hearing devices, by identifying new phenomena which they can exploit or mitigate and indicating how hearing aids might be designed to improve spatial hearing (goal #2)

C. Public Engagement: Our research will give new scientific knowledge of how spatial hearing works and how it is affected by hearing impairments or hearing aids. Our research will add to the wider public's knowledge of hearing. It will also encourage public awareness and discussion of just how important hearing impairment is as a health issue (goal #1).