Speech masking effects in speech communication across the lifespan

Lead Research Organisation: University College London
Department Name: Speech Hearing and Phonetic Science

Abstract

Our ability to communicate successfully with others can be strongly affected by the presence of noise and other voices in the environment, and children and older adults are more greatly affected than young adults in these situations. Even greater disruption is experienced for populations with hearing or language impairments, or even healthy adults who are non-native speakers. Some of the disruption is due to physical masking by interfering sounds (energetic masking EM) but if the disrupting sound can be understood, this also causes further difficulty (informational masking IM). Previous work suggests that informational masking causes relatively more disruption for children and older adults than for young adults but these findings are based on laboratory tests that are far from realistic communication.

Although the impact of adverse conditions on speech communication has been the object of studies in different age groups, no study to date has taken a full lifespan view, looking at the relative impact of IM and EM on participants aged from 8 to 85 using a common experimental design. Also, many studies have focused on the impact of EM and IM on speech perception using recorded sentences or words; this ignores the fact that speakers make dynamic adaptations in speech communication to counter the effects of masking. We evaluate the impact of adverse conditions in an interactive task using measures reflecting speech communication efficiency (e.g., task transaction time, rate of dysfluencies). Finally, there is little evidence to date as to whether laboratory-based evaluations reflect the level of difficulty experienced in everyday life. The proposed project will, for the first time, relate evaluations of speech communication difficulties in adverse listening conditions as measured in the laboratory with real-life ratings of difficulty collected in real time over a two week period. It will also test whether primarily informational masking causes greater interference for some age groups (e.g. children, older adults), and if the underlying reasons for the interference differ between children and adults.

In Study 1, 120 individuals aged 8 to 85 yrs will be tested in pairs while they carry out a problem-solving task in conditions varying in the degree of informational and energetic masking present. A secondary task will be added to make the task more cognitively demanding, thus reflecting real-life multitasking situations. Baseline measures of hearing, speech perception and cognitive function and a standardised questionnaire of auditory disability (SSQ) will also be collected. In Study 2, the same participants will be asked to report perceived communication difficulty and their listening environment on 6-7 occasions per day during a 2-week period using a smartphone-based app. Data from this study will be related to measures of communication effectiveness and SSQ data collected in Study 1. Finally, in Study 3, spontaneous speech will be collected for a group of 4-7 year olds using the same interactive task to complement measures from our previous corpora (and data for Study 1) and gain a full lifespan set of acoustic descriptors for conversational speech in good listening conditions.

This project will lead to a better understanding of how the impact of adverse listening conditions changes across the lifespan, of the relative effects of IM and EM in different age groups, evaluated in realistic communicative conditions, and of the true ecological validity of laboratory-based evaluations. It will also provide normative data for a set of acoustic-phonetic measures across a 4-85 year age range. This benchmark will be of use for practitioners such as speech and language therapists and audiologists who work on aspects of communication with individuals of all ages. Importantly, this research will also contribute to our basic understanding of speech perception and production development across the lifespan.

Planned Impact

Although this project is primarily a basic science project concerned with advancing our knowledge of the impact of masking on speech communication across the lifespan, we consider that the work has the potential to have impact in a number of fields. The main non-academic beneficiaries of this research will be audiologists, speech and language therapists (SLT), developers of speech technology applications. Finally a project leading to a better understanding of the causes of speech communication difficulties in noisy environments is of wider relevance to the general public.

Audiologists: It is well documented that speech communication in noisy environments is a main cause of difficulty for older listeners who are beginning to suffer age-related hearing loss as well as for any child or adult with hearing loss or auditory processing disorder. Although standardised audiometric tests of speech intelligibility in noise exist, they are limited to recorded words or sentences and have poor ecological validity as they do not include the adaptations that interlocutors would naturally make to counter masking effects in real-life interactions. Similarly, questionnaires asking a client to rate their auditory disability in different situations may under- or over-estimate actual difficulty. The findings of Study 1 will provide audiologists with a better understanding of speaker-related factors influencing the effect of different adverse conditions on speech communication in clients across the lifespan. The findings of Study 2, using the experiential sampling method (ESM), could be especially relevant, in terms of informing audiologists about the ecological validity of standardised tests of speech perception in noise. It will be a test also of the potential of ESM for clinical evaluations as alternatives to standardised tests and questionnaires.

Speech and Language Therapists: Although a wide range of instruments are available to SLTs for evaluating speech production and perception abilities in their clients in all age groups, these instruments do not directly test the ability to communicate in an ecologically-valid way, as they do not actually involve communication between two people or a situation in which clients are doing more than one task simultaneously. Our diapix method which is being developed in this project as a tool to gather measures of communication efficiency, rather than primarily as a means to elicit clear speech as previously used, has potential as such a tool especially given the recent development of the diapix app which gives automatic measures of transaction time.

Developers of speech technology: There is much current interest in trying to reduce the listening effort that results from communication in difficult conditions by automatically enhancing or 'enriching' the speech signal. Gaining a better understanding of how speaker-related factors affect such communication and of the relative impact of IM/EM masking in different age groups will help tailor such enrichment techniques for different populations of users.

Wider public: A great majority of the population experiences difficulty communicating in noisy environments, because of their age, their hearing, the fact that they are speaking in a foreign language, etc. The fact that speaking in the presence of other voices causes additional difficulty is not typically known even though it reflects the everyday listening situation of many people who may actually experience particular difficulties with informational masking (e.g, TV or radio in the background in many homes, as well as in care homes, background chatter in school environment). A better understanding of the impact of informational masking on particular age groups or populations could lead to simple adjustments being made to reduce such masking effects.
 
Description This project investigated the impact of different types of adverse environments on speech communication. The adverse conditions were either noise ('energetic masking') or three voices talking in the background ('informational masking'). Our main research aims were (a) to see whether these types of noise affect individuals differently according to their age and (b) whether difficulty in a setting simulating 'real life' communication correlated with estimates based on traditional laboratory-based tests. To evaluate communicative difficulty, we used a naturalistic task involving two participants completing a 'spot the difference' picture game (diapix) together while also doing a secondary task. This project is novel in investigating masking effects on speech communication across the lifespan (8-85 years) using communicative tasks rather than listening tests devoid of communicative intent. Also, we ran an additional study with the adult participants to relate the outcome of laboratory tests with real-time ratings of perceived communication difficulty in everyday environments using a smartphone app (EMA). A final benefit is to obtain acoustic-phonetic measures of spontaneous speech in casual and clear speaking styles across the lifespan.
We collected substantial data from 114 participants aged 8-85 years; all had normal hearing thresholds up to 4 kHz. In addition to diapix, participants completed a range of background tests evaluating their hearing and cognitive abilities. Results are still in the process of being fully analysed but findings suggest the following. From an analytic listening test, there is some evidence that hearing voices in the background interferes more with perception for children than for adults. This was not the case for background noise, which showed no age effect. In the diapix task, communication in adverse conditions increased ratings of listening effort at all ages; doing diapix with voices in the background was rated as more effortful than with noise; this was the case for all ages. Participants were also less accurate in the secondary task with voices rather than noise in the background; this reinforces the finding that informational masking causes greater interference and effort in communication than energetic masking, but without an obvious age effect. Acoustic measurements of speech recorded during diapix give further evidence of the impact of adverse conditions. When speech was in the background, there was increased vocal effort for children and 50-85 year olds; when noise was in the background, this was the case for 50-85 year olds only. In other speech production measures, there is less evidence of age effects. This was the case for the frequency of filled pauses ('hum', 'er'), countering our prediction of greater effects for older adults. Generally, ratings of effort were higher for the person taking the lead in the diapix task than for the person who had a passive role; this shows that even though most research is concerned with the impact of background interference on listening, speaking in adverse conditions has a strong impact on communication effort.
Further analyses are focusing on whether standardised analytic tests can help predict the level of difficulty experienced in more naturalistic communicative conditions.
Exploitation Route Reliable evaluations of speech communication difficulty are highly relevant for the clinical field of audiology and also for hearing aid developers. Traditionally, evaluations have been made using listening tests involving words or sentences but these are rather far from natural communication which involves communicative intent. There have been efforts in recent years to move towards more ecologically-valid evaluations of communication difficulty but this is still very much a field in development. Our methodology and key findings can inform the further development of ecologically-valid tests of speech communication. In terms of the academic impact, our key findings are relevant for researchers working in the field of hearing, speech perception, speech development and aging (e.g. effect of age on the impact of informational and energetic masking) and the field of speech production (e.g. effect of age on clear speech adaptations in adverse condition). Our findings are disseminated via journal publications and conference presentations.
Another significant output of this project has been the creation of a corpus of spontaneous speech dialogues for participants aged 8 to 85 (LifeLUCID) which is now available for download for research purposes (https://reshare.ukdataservice.ac.uk/854350/).
Sectors Healthcare,Pharmaceuticals and Medical Biotechnology

 
Title LifeLUCID Corpus: Recordings of Speakers Aged 8 to 85 Years Engaged in Interactive Task in the Presence of Energetic and Informational Masking 
Description This speech corpus contains recordings for 104 monolingual native southern British English speakers aged between 8 and 85 years old while they engaged in a problem-solving picture-based 'spot the difference' task (Diapix) with a conversational partner in four listening conditions. In NORM (quiet, no masking), participants heard each other normally. In SPSN (speech-shaped noise), participants communicated in speech shaped noise at an approximate signal-to-noise ratio of 0 dB. In IMRE (informational masking related picture), they communicated while three voices in the background talked about the same picture. In IMUR (information masking unrelated picture), they communicated while three voices in the background talked about a different Diapix picture. Participants were recorded in sex and age band matched pairs; they were unfamiliar with one another. They were seated in separate sound-treated booths and communicated via headphones. One participant in the pair took the lead in the interaction and participants switched roles, each completing four conditions in each role. For each recording, there is a stereo audio file (.wav format) with each participant on a separate channel and two textgrid files, with time-aligned orthographic transcripts for each of the participants. 
Type Of Material Database/Collection of data 
Year Produced 2021 
Provided To Others? Yes  
Impact The corpus has been downloaded by researchers external to UCL, and is also being used at UCL by Prof. Carolyn McGettigan for her research. 
URL https://reshare.ukdataservice.ac.uk/854350/
 
Description Collaboration with Professor Mary Rudner (University of Linkoping, Sweden) 
Organisation Linkoping University
Country Sweden 
Sector Academic/University 
PI Contribution Following a visit to the University of Linkoping to present my recent research on speech communication in older adults, we have further discussed the use of the diapix task to investigate speech communication in challenging conditions. I was invited to participate as co-investigator in three grant proposals to funding bodies in Sweden that will make use of our method to investigate speech communication in different populations (aging, immigrants of working age). One grant entitled 'Good auditory ecology for active and healthy aging' was funded for three years by Forte (Swedish Research Council for Health, Working Life and Welfare)
Collaborator Contribution This collaboration will also benefit our work, as the other co-investigator in these proposals, Professor Inga Holube, conducts studies involving Experiential Momentary Assessment, which is a technique that we are also using in our ESRC study. We will benefit from joint discussions on the use of these techniques.
Impact Three grant proposals to Swedish funding bodies on related research, in which i am named as co-investigator. One grant funded (see above)
Start Year 2019
 
Description Demonstration at evening event 'Easy Speaking Effortless Listening at Royal Institution 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact Our demonstration was part of an evening event at the Royal Institution focused on speech communication difficulties. This was aimed at members of the RI and the general public. A series of talks was followed by the opportunity for attendees to participate in hands-on demonstrations linked to this theme. We gave participants the opportunity to try out the diapix task and told them about our study. There were about 150 members of the public present. We had a leaflet available with further information about our study.
Year(s) Of Engagement Activity 2020
URL https://www.rigb.org/whats-on/events-2020/march/public-easy-speaking-effortless-listening
 
Description Invited talk at the University of Oregon, USA 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Postgraduate students
Results and Impact I was invited to give a talk entitled 'Clear speech strategies across the lifespan' in the Department of Linguistics at the University of Oregon in Eugene. I spent a day at the University having individual meetings with postgraduate students and faculty members.
Year(s) Of Engagement Activity 2018
URL https://blogs.uoregon.edu/gloss/2018/10/29/clear-speech-strategies-across-the-lifespan-by-dr-valerie...
 
Description Invited talk at the university of Linkoping, Sweden 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Postgraduate students
Results and Impact I was invided to the University of Linkping in Sweden to give a talk entitled 'The effect of aging and hearing status on speech communication in challenging conditions'. The talk was streamed via Twitter so that interested students and others unable to come to Linkoping could follow the talk and ask questions. An important outcome of the visit was to network with researchers based at the University that work on related research. This has led to me being included as collaborator in three grant proposals that were submitted in February 2018 to Swedish research councils.
Year(s) Of Engagement Activity 2018
 
Description Invited talk by Outi Tuomainen at University of Potsdam (Germany) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Postgraduate students
Results and Impact The outcome of our project was presented to students and staff from the University of Potsdam. This resulted in some interest in using our spontaneous speech elicitation techniques for a research project at the university.
Year(s) Of Engagement Activity 2020
 
Description Invited talk by Outi Tuomainen at the Freie University Berlin (Germany) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Postgraduate students
Results and Impact Outi Tuomainen gave an invited talk entitled "Acoustic-phonetic adaptations of spontaneous speech production in challenging conditions -a lifespan study" to students and staff from the Freie University Berlin. This gave exposure of our recent project to the research community in Berlin and engaged students in thinking about the use of new techniques for the study of speech communication.
Year(s) Of Engagement Activity 2021
URL https://www.ewi-psy.fu-berlin.de/einrichtungen/arbeitsbereiche/sonderpaed/Sprachentwicklung/Forschun...
 
Description Invited talk, conference 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact CHSCOM is a major conference on cognitive hearing science that attracts attendees from industry (hearing aid and cochlear implant companies) as well as academics and postgraduate students.
Year(s) Of Engagement Activity 2019
URL http://www.chscom2019.se/
 
Description Organisation of 2nd Workshop on Speech Perception and Production across the Lifespan 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact We organised what was due to be a two-day international workshop in London on 30-31 March 2020. Due to the restrictions imposed in the early stages of the Covid-19 pandemic, we had to very quickly rethink how to run this workshop successfully. We switched to an asynchronous online format at very short notice and managed to get recorded oral contributions or posters from a number of the 40 international contributors to the workshop. The book of abstract containing extended abstracts for all contributors was also published online. All materials remain publicly available on the workshop website.
Year(s) Of Engagement Activity 2020
URL https://www.sppl2020.org/