Decoding selective attention to speech for a mind-controlled hearing aid

Lead Research Organisation: Imperial College London
Department Name: Dept of Computing

Abstract

Current hearing aids do unfortunately not help with the most important problem that their users face: to understand speech in the presence of background noise such as other talkers. This is because current hearing aids amplify all sound in the environment, irrespective whether it is the speech that the user aims to listen to or background noise. This project will seek to develop mind-controlled hearing aids that selectively amplify the target speech and suppress background noise. To this end, the project will develop AI to decode, from brain recordings, which of several speech signals a hearing-aid user wants to attend. This signal can then be enhanced to boost its comprehension.

We will first acquire brain recordings in response to speech in different types of background noise. The brain responses will be measured non-invasively from electroencephalographic (EEG) recordings. These recordings employ scalp electrodes, and we will in particular consider Ear-EEG, that is, the recording of neural signals from the ear canal. This type of recording will allow the later integration of the developed technology with hearing aids that sit in the ear canal. The main part of the project will then consist of developing machine-learning methods for a reliable and real-time decoding of the attentional focus of the subject from the EEG data. To this end we will employ deep neural networks that will relate the spatio-temporal structure of the EEG data to characteristic aspects of speech, such as speech rhythms and pitch. The developed decoding strategies will then be tested in a prototype of a mind-controlled hearing aid, in which the amplification of different sounds will be controlled in real time by the brain recordings.

Publications

10 25 50

Studentship Projects

Project Reference Relationship Related To Start End Student Name
EP/S023283/1 01/04/2019 30/09/2027
2454811 Studentship EP/S023283/1 05/10/2020 04/10/2024 Michael Thornton