Interference in spoken communication: Evaluating the corrupting and disrupting effects of other voices

In everyday life, talking with other people is important not only for sharing knowledge and ideas, but also for maintaining a sense of belonging to a community. Most people take it for granted that they can converse with others with little or no effort. Successful communication involves understanding what is being said and being understood, but it is quite rare to hear the speech of a particular talker in isolation. Speech is typically heard in the presence of interfering sounds, which are often the voices of other talkers. The human auditory system, which is responsible for our sense of hearing, therefore faces the challenge of identifying which parts of the sounds reaching our ears have come from which talker.

Solving this "auditory scene analysis" problem involves separating those sound elements arising from one source (e.g., the voice of the talker to whom you are attending) from those arising from other sources, so that the identity and meaning of the target source can be interpreted by higher-level processes in the brain. Over the course of evolution, humans have been exposed to a variety of complex listening environments, and so we are generally very successful at understanding the speech of one person in the presence of other talkers. This contrasts with attempts to develop listening machines, which often fail when confronted with adverse conditions, such as automatic transcription of a conversation in an open-plan office. Human listeners with hearing impairment often find these environments especially difficult, even when using the latest developments in hearing-aid or cochlear-implant design, and so can struggle to communicate effectively in such conditions.

Much of the information necessary to understand speech (acoustic-phonetic information) is carried by the changes in frequency over time of a few broad peaks in the frequency spectrum of the speech signal, known as formants. The project aims to investigate how listeners presented with mixtures of target speech and interfering formants are able to group together the appropriate formants, and to reject others, such that the speech of the talker we want to listen to can be understood. Interfering sounds can have two kinds of effect - energetic masking, in which the neural response of the ear to the target is swamped by the response to the masker, and informational masking, in which the "auditory brain" fails to separate readily detectable parts of the target from the masker. The project will explore the informational masking component of interference - often the primary factor limiting speech intelligibility - using stimulus configurations that eliminate energetic masking. We will do so using perceptual experiments in which we measure how our ability to understand speech (e.g., the number of words reported correctly) changes under a variety of conditions.

The project will examine how acoustic-phonetic information is combined across formants. It will also explore how a speech-like interferer affects intelligibility, distinguishing the circumstances in which the interferer takes up some of the available perceptual processing capacity from those in which specific properties of the interferer intrude into the perception of the target speech. Our approach is to use artificial speech-like stimuli with precisely controlled properties, to mix target speech with carefully designed interferers that offer alternative grouping possibilities, and to measure how manipulating the properties of these interferers affects listeners' abilities to recognise the target speech in the mixture. The results will improve our understanding of how human listeners separate speech from interfering sounds and the constraints on that separation, helping to refine computational models of listening. Such refinements will in turn provide ways of improving the performance of devices such as hearing aids and automatic speech recognisers when they operate in adverse listening conditions.

Private-sector companies who develop hearing aids and cochlear implants:

Effective separation of a target voice from interfering speech is one of the main problems facing designers of hearing aids and cochlear implant (CI) processors. The project will provide greater understanding of how human listeners combine acoustic-phonetic information across formants and improved characterisation of the informational component of speech-on-speech interference. This will inform the development of enhanced hearing prostheses, and may potentially improve speech coding in CI processors. One of our Project Partners, Starkey, invests in longer-term issues in hearing research and is the largest manufacturer of hearing aids in the USA. We have also established contacts with the two major suppliers of hearing prostheses in the UK (Oticon and Phonak). Outcomes will be communicated to hearing aid and CI manufacturers by sending technical reports of our findings, by visits to selected companies, and meetings with their representatives at conferences.

Private-sector companies who develop robust automatic speech recognition (ASR) devices and techniques for speech enhancement:

Robust performance in the presence of interferers remains a key problem for ASR. The project will provide perceptual data for researchers in computational auditory scene analysis (CASA) to develop further models and algorithms for separating speech from interfering sounds; we already have established contacts with CASA researchers. This offers the prospect of improved front-end processors for ASR and speech enhancement systems, which is likely to improve the performance of commercial systems. Our other Project Partner, Audience, is one of the leading commercial developers of speech enhancement technology. Our research findings will be communicated to Audience via email and at regular meetings. A report will also be made available to other key companies in ASR and enhancement technology.

Charities that support the hearing impaired:

Our research findings will be communicated to the main UK charity, Action on Hearing Loss. Improved understanding of the perceptual processes underpinning successful separation of target speech from interfering speech will encourage the development of collaborative behavioural and modelling studies. Such cross-disciplinary interaction may stimulate funding of research projects on advanced hearing aids and CI algorithms likely to interest the charity. As well as articles in specialist journals, our findings will be communicated via a report tailored to the charity's interests.

The general public:

Indirectly, the impact on the above beneficiaries will also benefit the general public within 5-10 years. Improvements to hearing aid technology will benefit the ~10 million deaf and hard-of-hearing people in the UK. Similarly, there are ~320,000 CI users worldwide who would benefit from better techniques for encoding noisy speech in CI processors. Improved ASR and speech enhancement, which may arise from our project via its impact on CASA solutions, will enhance quality of life through improved speech-based communication with machines, and better electronically mediated vocal communication between individuals. The outcomes of the project will be communicated directly to the public via a lay summary on our project website, and through public engagement events.

The research fellow:

The RF will receive training on public engagement for researchers to facilitate his contributions to public lectures, report writing, and company visits. His involvement in writing journal articles and developing grant applications will put him in an excellent position for fellowship applications. His involvement in writing technical reports and visits to leading hearing-technology companies will foster links with the commercial sector likely to enhance his prospects of future employment either in academic or industrial research.