EnhanceMusic: Machine Learning Challenges to Revolutionise Music Listening for People with Hearing Loss

Every culture has music. It brings people together and shapes society. Music affects how we feel, tapping into the pleasure circuits of the brain. In the UK each year, the core music industry contributes £3.5bn to the economy (UK Music 2012) with 30 million people attending concerts and festivals (UK Music 2017). Music listening is widespread in shops, movies, ceremonies, live gigs, on mobile phones, etc.

Music is important to health and wellbeing. As a 2017 report by the All-Party Parliamentary Group on Arts, Health & Wellbeing demonstrates, "The arts can help keep us well, aid our recovery and support longer lives better lived. The arts can help meet major challenges facing health and social care: ageing, long-term conditions, loneliness and mental health. The arts can help save money in the health service and social care."

1 on 6 people in the UK has a hearing loss, and this number will increase as the population ages (RNID). Poorer hearing makes music harder to appreciate. Picking out lyrics or melody lines is more difficult; the thrill of a musician creating a barely audible note is lost if the sound is actually inaudible, and music becomes duller as high frequencies disappear. This risks disengagement from music and the loss of the health and wellbeing benefits it creates.

We need to personalise music so it works better for those with a hearing loss. We will consider:
1. Processing and remixing mixing desk feeds for live events or multitrack recordings.
2. Processing of stereo recordings in the cloud or on consumer devices.
3. Processing of music as picked up by hearing aid microphones.

For (1) and (2), the music can be broadcast directly to a hearing aid or headphones for reproduction.
For (1), having access to separate tracks for each musical instrument gives greater control over how sounds are processed. This is timely with future Object-Based Audio formats allowing this approach.
(2) is needed because we consume much recorded music. It's more efficient and effective to pre-process music than rely on hearing aids to improve the sound, as this allows more sophisticated signal processing.
(3) is important because hearing aids are the solution for much live music. But, the AHRC Hearing Aids for Music project found that 67% of hearing-aid users had some difficulty listening to music with hearing aids. Hearing aid research has focussed mostly on speech with music listening being relatively overlooked.

Audio signal processing is a very active and fast-moving area of research, but typically fails to consider those with a hearing loss. The latest techniques in signal processing and machine learning could revolutionise music for those with a hearing impairment. To achieve this we need more researchers to consider hearing loss and this can be achieved through a series of signal processing challenges. Such competitions are a proven technique for accelerating research, including growing a collaborative community who apply their skills and knowledge to a problem area.

We will develop tools, databases and objective models needed to run the challenges. This will lower barriers that currently prevent many researchers from considering hearing loss. Data would include the results of listening tests into how real people perceive audio quality, along with a characterisation of each test subject's hearing ability, because the music processing needs to be personalised. We will develop new objective models to predict how people with a hearing loss perceive audio quality of music. Such data and tools will allow researchers to develop novel algorithms.

The scientific legacy will be new approaches for mixing and processing music for people with a hearing loss, a test-bed that readily allows further research, better understanding of the audio quality required for music, and more audio and machine learning researchers considering the hearing abilities of the whole population for music listening.