Hearing in a social context: Understanding predictive mechanisms in communicative interaction

Lead Research Organisation: University of Nottingham
Department Name: School of Medicine

Abstract

Conversation is fundamental for developing social attachments, and if the ability to converse with others breaks down, it can be devastating. Not only does conversation with others improve psychological wellbeing, it also buffers people against the ill-effects of stress and other health issues. People with hearing impairment report social isolation and loneliness as one of its most disabling outcomes, yet we know very little about how to support conversation for people that struggle. To date, the majority of research examining speaking and listening behaviour has tested individuals in isolation, using experimental methods that do not require them to actually engage with someone else. But it is only by investigating speaking and listening in a social context that we can address the cognitive processes specific to conversation.

To interact smoothly, people must make mental predictions about each other's contributions. Conversations would be much less fluent if people waited for the person they were talking to to finish before they began preparing their response. Yet people with hearing impairment show reduced use of prediction during speech listening, even though they could particularly benefit from such help given their poor auditory input.

The objective of this project is therefore to understand how we make predictions during interaction, in order to identify ways of making conversation easier for people with hearing impairment. Combined behavioural and neuroscientific experiments will explore how people make predictions during interaction at both the mental level and the brain level, and I will draw this work together to generate a model of prediction in interaction. The way that predictions differ with hearing impairment will also be explored: do people with hearing impairment make predictions too late to be useful, are they less confident in their predictions, or are their predictions simply inaccurate? Following training with a hearing device manufacturer, subsequent work will identify how the findings could be implemented in hearing technology to support people with hearing impairment have conversations successfully. This work could transform the state-of-the-art in hearing devices, and therefore has the potential to benefit the psychological wellbeing of millions.

Planned Impact

I have identified two main beneficiaries for my 7-year research programme: (1) the hearing loss community, and (2) industries focused on interaction.

(1) Approximately one in six people in the UK have some form of hearing loss, which leads to loneliness and depression. Hearing devices offering improved comprehensibility for patients reduce such negative outcomes, and save businesses money in lost days and staff turnover. This research could therefore be used by hearing device manufacturers to motivate critical changes in the handling of the audio signal to support individuals with hearing loss. The patients themselves, their families, and the carers of individuals with hearing loss would benefit from such developments as it would improve engagement quality, and make it easier to share important information. While these stakeholders could benefit from the basic science findings of stage 1, I primarily aim to impact this group in stage 2. As the potential for quality of life benefit and increased wellbeing for a large population is significant, I will convey my research more widely by engaging with parliamentary interest groups (e.g. the All-Party Parliamentary Group on Deafness), and charities such as Action on Hearing Loss.

(2) This work could impact other industries focused on interaction, inspiring the development of new technology to improve the success and naturalness of their products. For example, teleconferencing systems can be problematic in noisy rooms. The hearing device algorithms developed in stage 2 could be incorporated into such systems to amplify relevant auditory information and attenuate irrelevant auditory information, improving user experience. In addition, my research could also be valuable to AI firms who work on predicting pathways through communication, as they could use my model of prediction (and relevant anonymised data) to train new algorithms.

Findings will be shared with these audiences through industry conferences/meetings, clinical workshops, engagement with parliament and charities, and press releases.
 
Description We have drawn together the vast work on face to face interaction, to review theories and methods and identify new ways forward for the field of communicative interaction (see Hadley et al., 2022). We have also published novel studies of communication behaviour, predictive behaviours in conversation, and perception of communication success.
Exploitation Route The hearing device industry has expressed interest in the work on communicative behaviours, and greater understanding of these behaviours and how they relation to communication success/hearing ability could be used to improve hearing devices in future.
Sectors Healthcare

Other

 
Description Contributed to the Roundtable on Research Bureaucracy
Geographic Reach National 
Policy Influence Type Contribution to a national consultation/review
 
Description MICA: Understanding and alleviating hearing disability: A cognitive-behavioural model of miscommunication in everyday conversation
Amount £2,090,500 (GBP)
Funding ID MR/X003620/1 
Organisation Medical Research Council (MRC) 
Sector Public
Country United Kingdom
Start 09/2022 
End 09/2027
 
Description Predicting language under difficult conditions: Effects of cognitive load, noise, and hearing impairment
Amount £690,049 (GBP)
Funding ID ES/X001148/1 
Organisation Economic and Social Research Council 
Sector Public
Country United Kingdom
Start 03/2023 
End 02/2026
 
Description Collaboration with Dr Jamie Ward (Goldsmiths) 
Organisation Goldsmiths, University of London
Country United Kingdom 
Sector Academic/University 
PI Contribution I brought access to data collected in my lab.
Collaborator Contribution Dr Ward contributed statistical analysis and intellectual input to a joint paper.
Impact Hadley, L. V., & Ward, J. A. (2021). Synchrony as a measure of conversation difficulty: Movement coherence increases with background noise level and complexity in dyads and triads. Plos one, 16(10), e0258247.
Start Year 2021
 
Description Collaboration with Dr Joseph Sollini (University of Nottingham) 
Organisation University of Nottingham
Country United Kingdom 
Sector Academic/University 
PI Contribution I contributed my psychology expertise and intellectual input to conduct a study of voice perception of older adults with and without hearing loss.
Collaborator Contribution Dr Sollini contributed expertise in psychophysics, and conducted analyses.
Impact We have a dataset of intelligibility of familiar vs unfamiliar speech in different levels of noise. We are running another group, then will put together a paper, then publish the dataset.
Start Year 2021
 
Description Collaboration with Prof Antonia Hamilton (UCL) 
Organisation University College London
Country United Kingdom 
Sector Academic/University 
PI Contribution I brought my expertise on communicative interaction to our collaboration, which involved co-writing a theoretical paper on nonverbal behaviour in face-to-face interaction.
Collaborator Contribution Prof Hamilton brought her expertise in social neuroscience to our collaborative paper.
Impact Hadley, L. V., Naylor, G., & Hamilton, A. F. D. C. (2022). A review of theories and methods in the science of face-to-face social interaction. Nature Reviews Psychology, 1(1), 42-54.
Start Year 2021
 
Description Collaboration with Prof John Culling (University of Cardiff) 
Organisation Cardiff University
Country United Kingdom 
Sector Academic/University 
PI Contribution I provided a dataset to explore in relation to a new research question: whether people switch their focus between talkers in a manner indicative of prediction during everyday conversation.
Collaborator Contribution Prof Culling brought expertise in analysis and modelling, to ask new questions of this dataset.
Impact We are currently writing up a paper on head movement in conversation.
Start Year 2021
 
Description Collaboration with Riikka Mottonen (University of Helsinki) 
Organisation University of Helsinki
Country Finland 
Sector Academic/University 
PI Contribution I brought equipment and expertise on speech prediction to the project.
Collaborator Contribution Riikka Mottonen trained myself and my staff on use of the TMS system, and provided analytical expertise for the project.
Impact There is a paper in preparation.
Start Year 2022
 
Description Collaboration with Ruth Corps (Nijmegen Max Planck) 
Organisation Max Planck Institute for Psycholinguistics
Country Netherlands 
Sector Public 
PI Contribution We brought facilities and access to a specialised participant population, as well as equipment for the studies.
Collaborator Contribution Ruth Corps brought expertise in the specific study method (verbal response collection and analysis).
Impact A study is currently running based on this collaboration.
Start Year 2023
 
Description Collaboration with Tim Beechey (University of Nottingham) 
Organisation University of Nottingham
Country United Kingdom 
Sector Academic/University 
PI Contribution I brought supervision experience to this partnership.
Collaborator Contribution Tim Beechey developed a novel task that we decided to develop into an electronic programme together with a student.
Impact None yet.
Start Year 2024
 
Description Blog about developing as an Early Career Researcher 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Postgraduate students
Results and Impact Wrote a blog on advice and lessons I've learned as an early career researcher. As a result, have been asked to mentor two other ECRs.
Year(s) Of Engagement Activity 2021
URL https://blogs.nottingham.ac.uk/medschoollife/2021/05/14/50-at-50-developing-as-an-early-career-resea...
 
Description Blog about fellowships (case study) 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Postgraduate students
Results and Impact Blog about early career researchers in academia. Received 4-5 emails from early career researchers with questions and wanting mentoring.
Year(s) Of Engagement Activity 2021
URL https://www.nottingham.ac.uk/research/research-areas/health-humanities/case-study.aspx
 
Description Early career researcher gala 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Other audiences
Results and Impact Led a full day event around how to develop as an early career researcher, including panel discussions, talks from industry, and poster competition.
Year(s) Of Engagement Activity 2021
 
Description Engaged with Festival of Science website 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact Wrote up basics of hearing and auditory processing for school-aged children, as well as related games and activities. Several hundred children/adults engage with this festival each year.
Year(s) Of Engagement Activity 2021
 
Description Public engagement day: Hearing behind the scenes 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Patients, carers and/or patient groups
Results and Impact Half day event for individuals with hearing loss to learn more about the research we conduct, and to learn about how studies are set up.
Year(s) Of Engagement Activity 2023
URL https://www.nottingham.ac.uk/research/groups/hs-predict/blogs/hearing-behind-the-scenes-public-engag...
 
Description Talk at Cardiff University 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Postgraduate students
Results and Impact Academic 45min talk on prediction, communication, and hearing. Dissemination of research. Led to introductions to other in the field.
Year(s) Of Engagement Activity 2021
 
Description Talk at Eriksholm 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Industry/Business
Results and Impact Talk to build relationships and enable future collaboration with the hearing device industry. Followups from three researchers in the industry signal interest in my work, with interest in a further industrial collaboration.
Year(s) Of Engagement Activity 2024
 
Description Talk at Erlanger Kolloquium 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact 20 minute talk on studying interaction in hearing research. Dissemination of research. Numerous questions from industry and academic attendees.
Year(s) Of Engagement Activity 2022
 
Description Talk at Goldsmiths College 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Postgraduate students
Results and Impact Academic 45min talk on prediction, communication, and hearing. Dissemination of research.
Year(s) Of Engagement Activity 2021
 
Description Talk at Kaiserslautern 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Postgraduate students
Results and Impact Talk at collaborator's university, including time with the relevant team.
Year(s) Of Engagement Activity 2023
 
Description Talk at ManCAD 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact Talk on conversation and predictive processing at talk series for hearing scientists. Approx 60 researchers and student attended.
Year(s) Of Engagement Activity 2022
 
Description Talk at Oldenburg 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Postgraduate students
Results and Impact Approx 40 researchers and student came to my presentation, and I have had continued contact with the department since.
Year(s) Of Engagement Activity 2022
 
Description Talk at the University of East London 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Postgraduate students
Results and Impact Academic 45min talk on prediction, communication, and hearing. Dissemination of research. Led to discussions around potential collaboration with the lab.
Year(s) Of Engagement Activity 2022
 
Description Talk at the University of Edinburgh 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Postgraduate students
Results and Impact Academic 45min talk on prediction, communication, and hearing. Dissemination of research.
Year(s) Of Engagement Activity 2021
 
Description Talk at the University of York 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Postgraduate students
Results and Impact Academic 45min talk on prediction, communication, and hearing. Dissemination of research. Led to 30min discussion with interested academics, and followup after.
Year(s) Of Engagement Activity 2022