Perceiving speech in single and multi-talker babble in normal and impaired hearing

Lead Research Organisation: University College London
Department Name: Speech Hearing and Phonetic Science

Abstract

Most speech is heard in the background of other sounds, particularly other people talking. Listeners with normal hearing have remarkable abilities to filter out extraneous sounds and listen only to the desired talker. This is known among researchers as the ?cocktail party effect?, because these abilities are so important at a noisy cocktail party. We are only beginning to understand the complex processes going on in the ears and brains of people as they perform this remarkable feat, far beyond anything current-day computers can manage. One fact is clear ? people with hearing impairment find this situation very challenging, even though they might function very well with their hearing aids in a quiet room. The situation is even worse for users of cochlear implants, devices which stimulate the hearing nerve directly with tiny electrical currents in cases where ordinary hearing aids are not adequate.
Any attempts to improve this situation through more sophisticated electronics in cochlear implants and hearing aids will depend on a thorough understanding of how the abilities of hearing-impaired listeners differ from those of normal listeners. In our laboratory model of a cocktail party (not including drinks!), both normal hearing and hearing-impaired participants will be tested on their perception of sentences in the presence of other background sounds. These other sounds will consist of a ?babble? of talkers in which the number of people talking simultaneously will vary from 1 to 16. We will test a number of ideas about why hearing-impaired people find this more difficult than normal listeners.
Our focus will be on one crucial factor ? ?babble? can fluctuate a lot in level, especially if there is only one other talker speaking. This creates ?dips? in the interfering sounds in which the target speech is more intense, thus easier to hear. Normal-hearing people can very effectively ?listen in the dips? of interfering sounds, but hearing-impaired listeners do this poorly. Many ideas have been advanced in explanation, including the way in which people with hearing impairment hear the pitch of voices, and their apparent inability to fuse together bits of sound information separated in time. What we hope to do is more fully explain how people with normal hearing manage this so well, whereas people with hearing impairment do not. We are hopeful that this deeper understanding will lead to new ideas for hearing aids that will enable hearing-impaired people to understand speech better in noisy environments.

Technical Summary

Understanding speech in the presence of masking sounds, such as other voices, is particularly challenging for those with impaired auditory systems. A major source of difficulty is a reduction or absence of masking release: unlike normally-hearing (NH) listeners, hearing-impaired (HI) listeners and cochlear implant (CI) users do not show substantially better speech perception with interfering sounds that fluctuate in level, such as speech, than with steady-state maskers. Attempts to identify the source of these deficits have been hindered by the fact that the processes underlying masking release in normal hearing are not completely understood, particularly in the ecologically valid situation where target speech is presented against a background of varying numbers of competing voices (n-talker babble). In NH listeners speech perception varies non-monotonically as the number of competing talkers increases from one to many, reflecting the interaction of different factors that have not been isolated in previous research, including: opportunities to glimpse the target afforded by fluctuations in masker level; difficulties in attending to the target rather than the masking speech; and interference in auditory object formation arising from the presence of multiple fundamental frequency (F0) contours. A major element of the proposed work is the development of novel stimulus configurations to allow independent assessment of the role of such factors in changes in performance with increasing numbers of interfering talkers. It is anticipated that the contribution of different factors will vary across different listener groups. Possible limitations on CI and HI listeners? ability to benefit from glimpsing will be examined with specific investigations of 1) the role of periodicity, including consideration of whether it is cued by temporal fine structure or envelope cues; 2) deficits in the ability to fuse segments of speech information over brief temporal gaps; and 3) effects of overall signal-to-noise ratio on the utility of glimpsing. Other issues to be addressed include effects of the presence of multiple F0 contours in the masker, and the role of masker intelligibility. The latter will be investigated using novel stimuli that allow comparison between intelligible and unintelligible maskers that both incorporate the spectro-temporal complexity of speech. The proposed work will both extend our basic scientific knowledge of speech perception in noise and also have important practical applications in the development of technology to improve outcomes for CI and hearing-aid users, and the enhancement of diagnostic tests probing disabilities such as auditory processing disorder.

Publications

10 25 50
 
Description DTU SCA - SMALL PROJECTS
Amount £5,000 (GBP)
Organisation Technical University of Denmark 
Sector Academic/University
Country Denmark
Start 03/2017 
End 06/2017
 
Title AzBio Sentence Corpus 
Description British English versions of the AzBio sentence materials originally developed in the USA for testing cochlear implant recipients. Recordings of 500 sentences obtained in an anechoic chamber from one male speaker of Southern British English 
Type Of Material Database/Collection of Data/Biological Samples 
Provided To Others? No  
Impact To be used in ongoing studies 
 
Title BEL sentence corpus 
Description British English versions of the Basic English Lexicon sentences. Recordings of 500 sentences made in an anechoic chamber from two male and one female speaker of Southern British English. 
Type Of Material Database/Collection of Data/Biological Samples 
Provided To Others? No  
Impact Currently being used in ongoing studies 
 
Title Speech Test Corpus 
Description Recordings made in an anechoic chamber from one male and one female speaker of Southern British English of the following materials: 720 IEEE sentences 336 BKB sentences 456 monosyllabic words 72 VCV (vowel-consonant-vowel) stimuli comprising all 24 English consonants combined with 3 different vowels 
Type Of Material Improvements to research infrastructure 
Provided To Others? No  
Impact Currently being used in active studies. 
 
Title Vocoding software 
Description Implementation of methods to process speech signals using a vocoder in which the excitation can be varied in many ways, with a particular emphasis on the manipulation of the phase relationships in the harmonic complex. 
Type Of Material Improvements to research infrastructure 
Year Produced 2012 
Provided To Others? Yes  
Impact Currently being used in active studies. Also used for studies resulting in the publication: Green, T. & Rosen, S. (2013) Phase effects on the masking of speech by harmonic complexes: Variations with level. J Acoust Soc Am. 134(4): 2876-2883. 
 
Title Whispered BEL sentence corpus 
Description Whispered versions of anglicised Basic English Lexicon sentences. Recordings of 500 sentences made in an anechoic chamber from one male and one female speaker of Southern British English. 
Type Of Material Improvements to research infrastructure 
Year Produced 2015 
Provided To Others? No  
Impact To be used in ongoing studies 
 
Title Whispered CV corpus 
Description Two talkers recorded, in an anechoic recording chamber, a set of vowel-consonant-vowel (VCV) utterances comprising all combinations of 3 vowels and all 24 English consonants multiple times. From these were selected two 'good' tokens from each of the two talkers. A corresponding set of normally-voiced VCVs were obtained from the same 2 talkers for comparison purposes. 
Type Of Material Database/Collection of Data/Biological Samples 
Provided To Others? No  
Impact Not yet. 
 
Description E-LOBES (Environment-aware Listener-Optimized Binaural Enhancement of Speech). 
Organisation Imperial College London
Country United Kingdom 
Sector Academic/University 
PI Contribution We are providing expertise concerning human auditory functioning, speech in noise perception, the properties of impaired hearing, and intelligibility modelling.
Collaborator Contribution They provide start of the art signal processing techniques.
Impact None as yet
Start Year 2015
 
Description INSPIRE: Marie Curie Initial Training Network 
Organisation Cochlear Ltd.
Country Global 
Sector Private 
PI Contribution We contributed to the proposal leading to the award of this FP7 Marie Curie Initial Training Network 'Investigating Speech Processing In Realistic Environments (INSPIRE) . We supervise one fully-funded PhD student, and will have others visiting our laboratory as part of this programme.
Collaborator Contribution Our student will visit DTU for 3-6 months, doing a project there, as well as participate in the activities of at least one non-academic unit, a hospital or commercial company. Twice-yearly scientific meetings aim to more broadly educate the students. The work of our student fits closely within the themes of the MRC grant.
Impact One PhD student recruited and in place in September 2012.
Start Year 2012
 
Description INSPIRE: Marie Curie Initial Training Network 
Organisation Radboud University Nijmegen
Department Centre for Language and Speech Technology
Country Netherlands 
Sector Academic/University 
PI Contribution We contributed to the proposal leading to the award of this FP7 Marie Curie Initial Training Network 'Investigating Speech Processing In Realistic Environments (INSPIRE) . We supervise one fully-funded PhD student, and will have others visiting our laboratory as part of this programme.
Collaborator Contribution Our student will visit DTU for 3-6 months, doing a project there, as well as participate in the activities of at least one non-academic unit, a hospital or commercial company. Twice-yearly scientific meetings aim to more broadly educate the students. The work of our student fits closely within the themes of the MRC grant.
Impact One PhD student recruited and in place in September 2012.
Start Year 2012
 
Description INSPIRE: Marie Curie Initial Training Network 
Organisation Royal National Throat, Nose and Ear Hospital
Department Department of Audiology
Country United Kingdom 
Sector Hospitals 
PI Contribution We contributed to the proposal leading to the award of this FP7 Marie Curie Initial Training Network 'Investigating Speech Processing In Realistic Environments (INSPIRE) . We supervise one fully-funded PhD student, and will have others visiting our laboratory as part of this programme.
Collaborator Contribution Our student will visit DTU for 3-6 months, doing a project there, as well as participate in the activities of at least one non-academic unit, a hospital or commercial company. Twice-yearly scientific meetings aim to more broadly educate the students. The work of our student fits closely within the themes of the MRC grant.
Impact One PhD student recruited and in place in September 2012.
Start Year 2012
 
Description INSPIRE: Marie Curie Initial Training Network 
Organisation Technical University of Denmark
Department Department of Electrical Engineering
Country Denmark 
Sector Academic/University 
PI Contribution We contributed to the proposal leading to the award of this FP7 Marie Curie Initial Training Network 'Investigating Speech Processing In Realistic Environments (INSPIRE) . We supervise one fully-funded PhD student, and will have others visiting our laboratory as part of this programme.
Collaborator Contribution Our student will visit DTU for 3-6 months, doing a project there, as well as participate in the activities of at least one non-academic unit, a hospital or commercial company. Twice-yearly scientific meetings aim to more broadly educate the students. The work of our student fits closely within the themes of the MRC grant.
Impact One PhD student recruited and in place in September 2012.
Start Year 2012
 
Description APD information day 2015 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Patients, carers and/or patient groups
Results and Impact I participated as speaker and discussant for an Auditory Processing Disorder (APD) information day. Of 130 attendees, about 80 were families, with the rest clinicians. I spoke generally about the problems of understanding speech in noise and how problems could arise.
Year(s) Of Engagement Activity 2015
 
Description BSHAA 2015 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact I presented a talk concerning the problems of hearing-impaired people understanding speech-in-noise to the British Society of Hearing Aid Audiologists (BSHAA) for its Professional Development Day in March 2015 (~50 participants). The BSHAA is the professional body for the independent hearing aid profession in the UK. Of the 33 participants who provided feedback concerning the Professional Development Day mentioned above, 26 thought Prof Rosen's talk highly relevant to their practice while 6 thought it mostly relevant. Comments on the day of talks included: This was one of the best study days I have ever attended; Highly relevant and interesting; Excellent meeting, very relevant and pro-active; A big improvement over the last meeting! ; Very good speakers.
Year(s) Of Engagement Activity 2015
 
Description CAPD Ireland 2015 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Invited keynote speaker to "Developments in Central Auditory Processing Disorder (CAPD): Implications for Ireland, Dublin", organised by CAPD Ireland primarily for clinicians and spoke about the problems of understanding speech in 'noise'. 20 of the 55 feedback forms returned specifically mentioned my talk as being particularly enlightening.
Year(s) Of Engagement Activity 2015
 
Description Material World appearance 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact Arising from the Royal Society Exhibition, 'Speaking and Listening in a Noisy World', I was interviewed for the BBC Radio 4 programme 'Material World' about the work represented at the stand.

http://www.bbc.co.uk/programmes/b01292vf

None
Year(s) Of Engagement Activity 2011
 
Description Participation in a colloquium aimed at clinicians 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Invited speaker to a colloquium on audiology organised by the Institut libre Marie Haps, Bruxelles, Comprendre l'intelligibilité dans le bruit (Understanding speech in noise), aimed at practicing and trainee audiologists and medical professionals, talk entitled: Understanding speech in the background of other sounds.
Year(s) Of Engagement Activity 2017
URL http://www.vinci.be/fr-be/mariehaps/Pages/Colloque-en-audiologie--.aspx
 
Description Royal Institution (Ri) PE event 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact On behalf of the INSPIRE project, an EU Initial Training Network, and influenced by my MRC research project, I organised an evening at the Royal Institution (Ri) entitled Good listeners and smooth talkers: Spoken communication in a challenging world. This consisted of 4 short lectures (in which I gave one plus the introductory speech) followed by about 20 interactive demonstrations which involved participants from UCL and 7 European countries. Over 250 people paid to attend this event.

Students gained experience in presenting research results to a lay audience
Year(s) Of Engagement Activity 2015
URL http://www.rigb.org/whats-on/events-2015/january/public-good-listeners-and-smooth-talkers
 
Description Royal Society Summer Science Exhibition 2011 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? Yes
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact I spearheaded the proposal and development of "Speaking and Listening in a Noisy World", which was one of 22 exhibits chosen by the Royal Society across all scientific disciplines to show-case U.K research at the 2011 Summer Exhibition of Science from 5-10th July. Over 13,000 visitors attended the exhibition.

The exhibit brought together scientists from across four UCL Departments and disciplines and a large body of PhD and MSc students required to facilitate the exhibit across the five days.

The exhibit was targeted at adults and children; its main objective was to provide an overview of current UCL research on the effect of noise and other types of degradation on speech communication in an engaging and stimulating fashion. This was achieved using a number of interactive demonstrations, quizzes and puzzles that were specially developed for this exhibit.

http://royalsociety.org/summer-science/2011/noisy-world/


A number of media stories were generated from this exhibit. In particular, I was interviewed for the BBC Radio 4 programme 'Material World' about the work represented at the stand.

http://www.bbc.co.uk/programmes/b01292vf
Year(s) Of Engagement Activity 2011
URL http://royalsociety.org/summer-science/2011/noisy-world/
 
Description Two tutorial talks for audiologists 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Invited keynote speaker (two separate talks) to the 55th Annual Conference of the Israeli Speech Hearing and Language Association, consisting primarily of audiologists. Addressed issues of perceiving speech in noise in a wide variety of populations, and also the roles of envelope and temporal fine structure in the understanding of auditory perception.
Year(s) Of Engagement Activity 2019
URL http://www.congress.co.il/ishla/