Predicting language under difficult conditions: Effects of cognitive load, noise, and hearing impairment
Lead Research Organisation:
University of Edinburgh
Department Name: Sch of Philosophy Psychology & Language
Abstract
Understanding speech is extraordinarily complex, yet people can generally understand what somebody has said as soon as they hear it. They manage to do so, in part, because they predict what the speaker is going to say next. For example, if a speaker says "I would like to go outside to fly a...", then the listener may predict that the speaker's next word will likely be "kite". Listeners use these predictions to "get ahead of the game", making comprehension more rapid and efficient. But we do not know what happens when listeners are unable to make these predictions (because their efforts are elsewhere), such as when they have to concentrate hard to hear the utterance, when they are paying attention to another task, or when they have a hearing impairment. In fact, given that people with hearing impairment already receive degraded auditory information and have to apply extra effort to understand what is being said, they could particularly benefit from predictions to "stay on the front foot". In this project, we investigate the factors that disrupt prediction when listening to speech to better understand the how prediction works in listeners with and without hearing impairment. By identifying the types of difficulty that impair prediction, we can start to identify methods to help resolve these difficulties and improve speech understanding, both for people with normal hearing in challenging listening situations, and for older adults with hearing impairment.
We use a well-established technique known as the visual-world paradigm, in which participants listen to sentences and view objects on a screen while their eye movements are recorded. In previous studies, people heard (for example) the sentence "I would like to wear..." while seeing pictures of a tie, a dress, a drill, and a hairdryer. They tended to look at pictures of wearable objects (i.e., the tie or dress) before the speaker actually named the object, and they did so very quickly. We call this associative prediction, because it is dependent on simple associations with the word "wear". However, our work has shown that people also take their knowledge of the speaker into account when predicting. For example, they tended to look at the picture of a tie (a stereotypically masculine object) when they heard a male speaker say "I would like to wear...", but the picture of a dress (a stereotypically feminine object) when it was said by a female speaker. We call this strategic prediction, because it requires the listener to additionally take the perspective of the speaker. These strategic predictions are particularly valuable because they more accurately estimate what the speaker is likely to say, but they are slow and effortful.
Our goal is to investigate the factors that hinder both simple associative predictions, and more complex strategic predictions. In a series of experiments, participants perform the visual-world paradigm under conditions of hearing difficulty (by listening to sentences in background noise) and increased task difficulty (by completing a second task while listening). Some of our experiments use young participants with normal hearing (conducted at the University of Edinburgh). Other experiments use older participants with and without hearing impairment (conducted at Hearing Sciences - Scottish Section, part of the University of Nottingham located in Glasgow). The studies address the following questions: (1) How do adverse listening conditions and increased task difficulty affect the two types of prediction?; (2) Does difficulty affect people with and without hearing impairment in the same or different ways?; and (3) Are the effects of hearing loss comparable to perceptual difficulty, cognitive difficulty, or a combination of both?
Answering these questions will allow us to identify methods for overcoming such difficulties to improve speech understanding, particularly for people who struggle because of hearing loss.
We use a well-established technique known as the visual-world paradigm, in which participants listen to sentences and view objects on a screen while their eye movements are recorded. In previous studies, people heard (for example) the sentence "I would like to wear..." while seeing pictures of a tie, a dress, a drill, and a hairdryer. They tended to look at pictures of wearable objects (i.e., the tie or dress) before the speaker actually named the object, and they did so very quickly. We call this associative prediction, because it is dependent on simple associations with the word "wear". However, our work has shown that people also take their knowledge of the speaker into account when predicting. For example, they tended to look at the picture of a tie (a stereotypically masculine object) when they heard a male speaker say "I would like to wear...", but the picture of a dress (a stereotypically feminine object) when it was said by a female speaker. We call this strategic prediction, because it requires the listener to additionally take the perspective of the speaker. These strategic predictions are particularly valuable because they more accurately estimate what the speaker is likely to say, but they are slow and effortful.
Our goal is to investigate the factors that hinder both simple associative predictions, and more complex strategic predictions. In a series of experiments, participants perform the visual-world paradigm under conditions of hearing difficulty (by listening to sentences in background noise) and increased task difficulty (by completing a second task while listening). Some of our experiments use young participants with normal hearing (conducted at the University of Edinburgh). Other experiments use older participants with and without hearing impairment (conducted at Hearing Sciences - Scottish Section, part of the University of Nottingham located in Glasgow). The studies address the following questions: (1) How do adverse listening conditions and increased task difficulty affect the two types of prediction?; (2) Does difficulty affect people with and without hearing impairment in the same or different ways?; and (3) Are the effects of hearing loss comparable to perceptual difficulty, cognitive difficulty, or a combination of both?
Answering these questions will allow us to identify methods for overcoming such difficulties to improve speech understanding, particularly for people who struggle because of hearing loss.
