New pathways to hearing: A multisensory noise reducing and palate based sensory substitution device for speech perception

Disabling hearing loss is a global problem that affects nearly half a billion people. Furthermore, it is a problem that is growing with an aging population and has clear negative functional, social, emotional, and economic impacts. In the United Kingdom, adult onset hearing loss is predicted to be one of the top ten disease burdens by 2030 (WHO). Commercially available correction for hearing loss is mostly limited to hearing aids and cochlear implants. These devices suffer from signal processing and sensory transduction limitations. On the signal processing side, they struggle with the separation of speech from noise, often from other voices in social situations - the cocktail party phenomenon. On the transduction side, devices continue to rely on the damaged cochlea as the channel of communication. The aim of this proposal is to address these limitations through multisensory remapping at both the signal processing and transduction stages.
We will address signal processing limitations by introducing a new multisensory algorithm that will aim to recover the auditory signal from talking faces. The moving face can provide a source of information that is independent of environmental noise. Facial movement can also be used to enhance the signal to noise ratio of audio-only based speech. The new method can also recover facial movement from auditory signals alone so that speech perception might continually benefit from known improvements associated with being able to see the face, even when it is not present, as when the recovered face is presented on a device carried by the listener (e.g. a smart phone or Google Glass).
We will address signal transduction limitations by building on recent successes in supplementing vision through high-density tactile stimulation of the tongue and previous work demonstrating promise for supplementing word learning through tactile stimulation. In particular, we will build a novel non-invasive conformable high-density electrode array that provides electrotactile stimulation of the hard palate. This is the first device with high enough channel density to realistically provide, in tactile form, the spatial information about sound frequency available along the healthy cochlea. By putting it on the hard palate, this device will be the first sensory supplementation device to have direct access to sensorimotor brain circuitry important for speech learning and perception through the trigeminal nerve.
Finally, we will use behavioural and brain imaging methods to experimentally test the combined use of these signal processing and transduction innovations for hearing supplementation. From past experience with more primitive devices (e.g. The Tickle Talker), we expect people will be able to rapidly learn words and transfer training to novel words in new contexts. We expect training to be enhanced by combined presentation of audio to the hard palate and the face to a portable display device, so learning can occur in natural contexts. We will test the ability of participants to use the device for speech perception behaviourally and use functional imaging to look for indications of activation or modification of speech circuits in the brain after training. This work will contribute to our understanding of multisensory signal processing algorithms for hearing devices, auditory-to-tactile hearing supplementation, and multisensory brain plasticity. Success with this experimental device would warrant clinical trials to supplement hearing in individuals with hearing loss and bring these innovations to market as a new device to help with the social and economic challenges posed by disabling hearing loss.

Disabling hearing loss is a global problem that affects nearly half a billion people. More than 16% of the UK population overall and 70% of those over 70 has some form of hearing loss (Action on Hearing Loss Statistics, 2014). The problem is particularly acute given the ageing population and the ubiquity of age-related hearing loss. Age-related hearing loss is one of the most significant contributors to social isolation and lack of wellbeing in the elderly. Associated problems include increased social and cognitive difficulties, social isolation, depression, dementia, and mortality when uncorrected (Arlinger, S., Negative consequences of uncorrected hearing loss-a review. International Journal of Audiology, 2003). Commercially available correction for hearing loss is mostly limited to hearing aids that only amplify sounds and cochlear implants that are invasive, expensive, and limited in the number of channels carrying acoustic information. Health related quality of life and cost effectiveness results associated with these devices are quite variable with only small to medium typical effect sizes (e.g., Chisolm et al., 2007). The technologies on which these devices are built are already mature. Our strategy is to take a different approach that exploits recent advances in computation power, particularly in handheld devices, and new developments in the fabrication of conformable electrode arrays. Our project makes two primary advances that will have an impact on those with hearing loss. First, we propose a new multisensory signal processing algorithm for increasing the signal to noise ratio of speech through facial speech reading that can be adopted by both existing hearing devices and a new generation of devices. Second, we propose a new signal transduction method for supplementing speech hearing through a high-density conformable electrode array worn on the hard palate. We have identified a project partner Wicab Inc. who already market an electrotactile stimulator for the blind which is placed on the tongue. They have expressed a strong interest in their letter of support in developing new products for the deaf and deaf-blind and are keen to support the project with their existing expertise and to develop any new devices to a point at which they could be brought to market.