Listening effort and multilingual speech communication: Neural measures of auditory and lexical processing by adults and older children

People feel like they need to "listen harder" when communicating in a second language, but it isn't clear how this effort changes the brain processes involved in recognising speech. Our initial research has produced a surprising finding; we tested people who were trying to listen to a talker in a noisy background (i.e., a distracting talker), and found that auditory areas of the brain are better at picking out the target talker when people are listening to a second language than their first language. We did this by recording neural activity (electroencephalography; EEG) and measuring how it becomes entrained to the acoustics of speech. Although people would normally be expected to perform better when listening to their first language, we think that second-language listeners had more selective auditory processing because of their additional listening effort. We found related effects for neural measures of word recognition in the same task, and think that we've found mechanisms that allow second-language learners to partially compensate for their speech recognition difficulties. In this grant project, we will expand our investigation in a series of studies that manipulate the acoustics of speech, and compare how speech is recognised in first and second languages by speakers of English and Korean. Furthermore, we will test adults who learned both languages at the same time when they were young children, adults who learned their second language later in life, and older children who are in the process of learning both languages. Our goals are to understand how people can use listening effort to compensate for their difficulties with second-language speech, and examine how this ability develops and relates to proficiency. This work is important for understanding how people apply their processes and structures for language during everyday speech communication, and is relevant to a wide range of difficult listening conditions (e.g., hearing impairment). The work will also advance our scientific understanding of how new measures of neural entrainment for speech relate to practical aspects of speech recognition.

Our research will be of interest to a variety of non-academic audiences, including the general public and society; multilingual teachers, parents and children; and hearing device manufacturers.

General public and society. Language learning, speech, development, and neuroscience are topics that are of great popular interest, with relevance to how our multilingual society affects individuals. We have been active in public engagement, with frequent media contacts as well as participation in public science events, and will include the results of this project in these engagement activities.

Multilingual teachers, parents and children. Our outreach and recruitment activities will involve multilingual teachers, parents and children, and we expect them to be strongly interested in how multilingualism affects individuals and their education. We will educate them about broader issues related to development, education, bilingualism, and neuroscience, in addition to disseminating our project results.

Hearing device manufacturers. Listening effort and cognitive load are of current interest to hearing device manufacturers (e.g., hearing aids and cochlear implants); they are working to design less tiring devices as well as exploring how effort and load can be quantified. We will disseminate our findings on how to quantify effort and load at auditory and lexical levels, and how these effects differ for adults and children.