Challenges To Revolutionise Hearing Device Processing
Lead Research Organisation:
University of Nottingham
Department Name: School of Medicine
Abstract
Abstracts are not currently available in GtR for all funded research. This is normally because the abstract was not required at the time of proposal submission, but may be because it included sensitive information such as personal details.
Organisations
Publications
Akeroyd M
(2020)
Launching the first "Clarity" Machine Learning Challenge to revolutionise hearing device processing
in The Journal of the Acoustical Society of America
Akeroyd M
(2024)
The Clarity & Cadenza Challenges
Akeroyd M
(2023)
Results of the second "clarity" enhancement challenge for hearing devices
in The Journal of the Acoustical Society of America
Akeroyd M.A.
(2022)
Predicting Speech Intelligibility for People with Hearing Loss: The Clarity Challenges
in Internoise 2022 - 51st International Congress and Exposition on Noise Control Engineering
Cox T
(2023)
Predicting Speech Intelligibility for People with a Hearing Loss: The Clarity Challenges
in INTER-NOISE and NOISE-CON Congress and Conference Proceedings
| Description | We have run four competitions aimed at improving the signal processing in hearing aids. In each competition we supply a very large number of speech-in-noise signals, representing someone listening to a talker in a room but made difficult by there being other background sounds on at the same time, plus software for modelling what a hearing aid will do to the sounds and predicting how many words will be heard. Any research group around the world can enter; the challenge is to improve the number of words or how well they can match what real listeners do. Each competition is designed to be a harder situation than the preceding one. |
| Exploitation Route | They will be useful to hearing-aid developers and also those interested in auditory theory and how hearing problems affect listening. They can allow companies developing speech technologies to make those accessible to those with hearing loss. |
| Sectors | Digital/Communication/Information Technologies (including Software) Healthcare |
| URL | https://2023.speech-in-noise.eu/?p=programme&id=21 |
| Description | We organised the first open Machine Learning Challenge for enhancing the processing of hearing aids in 2021. This was part of developing a new research area and growing the number of researchers considering hearing loss and speech technology. Non-academic entrants were from: Music Tribe; Google and Tancent. Academic entrants were: Brno University of Technology; ELO-SPHERES Consortium; University of Oldenburg; University of Sheffield; University Hannover and Shenzhen University. The results were presented at the The Clarity Workshop on Machine Learning Challenges for Hearing Aids (September 2021). We also organised the first open Speech Intelligibility Prediction Challenge for people with a hearing loss listening via hearing aid. Ten entrants came from a variety of universities in Taiwan, China, Japan, Germany and UK. The results were presented at the The 2nd Clarity Workshop on Machine Learning Challenges for Hearing Aids (June 2022). Next. we organised the second open Machine Learning Challenge for enhancing the processing of hearing aids in 2022. The second challenge substantially increased the difficulty of the problem by including multiple interferers and head movement during the stimulus. This second challenge was also used as the basis for an ICASSP Signal Processing grand challenge in 2022/23. The ICASSP challenge also included a more ecologically-valid and independent evaluation set. There were 10 teams between these two challenges. The results were presented at the 2022 2nd Clarity Workshop on Machine Learning Challenges for Hearing Aids (December 2022) at at ICASSP. |
| First Year Of Impact | 2022 |
| Sector | Digital/Communication/Information Technologies (including Software),Healthcare |
| Impact Types | Cultural |
| Description | EnhanceMusic: Machine Learning Challenges to Revolutionise Music Listening for People with Hearing Loss |
| Amount | £1,319,161 (GBP) |
| Funding ID | EP/W019434/1 |
| Organisation | Engineering and Physical Sciences Research Council (EPSRC) |
| Sector | Public |
| Country | United Kingdom |
| Start | 06/2022 |
| End | 12/2026 |
| Title | 1st Clarity Prediction Challenge (CPC1) |
| Description | The 1st Clarity Prediction Challenge is a standardised set of data and tools that have been published to allow the comparison of approaches to hearing aid intelligibility prediction. The data includes a set of over 10,000 responses from a panel of hearing-impaired listeners who were asked to repeat what they heard when presented with hearing aid signals containing spoken utterances. Both the 'ground truth' transcription (i.e. the true sequence of words in the utterance) and the listener's responses are included. The data can be used for testing intelligibility prediction models. The data is provided alongside a standardised prediction task with clearly defined training and evaluation datasets, and prediction evaluation metrics. Tools are provided to ensure that the evaluation metrics are correctly applied. The tools and data have been used for a formal evaluation which attracted a good number of high-quality submissions. Results of this evaluation are published so that users of the data have external benchmarks against which to compare their novel approaches. |
| Type Of Material | Improvements to research infrastructure |
| Year Produced | 2022 |
| Provided To Others? | Yes |
| Impact | The tools and data were made publically available and researchers were invited to use it to compare their hearing aid intelligibility prediction algorithms. A number of teams from the US, Europe and Asia participated representing both academia and industry. Initial results were announced at a virtual workshop event, and the best systems were invited to participate in a special session that was held at the leading international speech conference, INTERSPEECH 2022 |
| URL | https://claritychallenge.org/docs/cpc1/cpc1_intro |
| Title | 2nd Clarity Enhancement Challenge (CEC2) |
| Description | The 2nd Clarity Enhancement Challenge is a standardised set of data and tools that has been published to allow the comparison of hearing aid processing algorithms. The data includes simulated hearing aid inputs that model a situation that hearing aid users find challenging: a target sentence spoken in a domestic living room while multiple speech, music and noise source are active in the background. CEC2 is an extension of our previously released CEC1. It simulates more complex listening environments and uses ambisonic processing to simulate head-movement. The data has been published alongside a set of tools for building baseline enhancement systems, and software to implement standard hearing aid speech intelligibility and quality evaluation metrics. In addition to the simulated data, we have also provided a further set of 'real' signals that are based on the same scenario but which employ actors talking to distant microphones. As with our previous dataset (CEC1), the data and tools have been used in international evaluations with published results, meaning that future users of the data will have state-of-the-art results against which to benchmark their approaches. |
| Type Of Material | Improvements to research infrastructure |
| Year Produced | 2023 |
| Provided To Others? | Yes |
| Impact | The tools and data were made publically available and researchers were invited to use it to compare their hearing aid speech enhancement algorithms. Two challenge rounds have been organised. In the first, systems were evaluated with a hearing-impaired listening panel and results were presented at an online workshop. The second round, using the 'real' data for evaluation was run as an ICASSP Signal Processing Grand Challenge and will feature in a special session at ICASSP 2023. |
| URL | https://claritychallenge.org/docs/icassp2023/icassp2023_intro |
| Title | 2nd Clarity Prediction Challenge (CPC2) |
| Description | he 2nd Clarity Prediction Challenge (CPC2) is a standardised set of data and tools that have been published to allow the comparison of approaches to hearing aid intelligibility prediction. The data extends the previously released CPC1 dataset to include 20,000 responses from a panel of hearing-impaired listeners who were asked to repeat what they heard when presented with hearing aid signals containing spoken utterances. Both the 'ground truth' transcription (i.e. the true sequence of words in the utterance) and the listener's responses are included. The data can be used for testing intelligibility prediction models. The data is provided alongside a standardised prediction task with clearly defined training and evaluation datasets, and prediction evaluation metrics. Tools are provided to ensure that the evaluation metrics are correctly applied. The tools and data have been used for a formal evaluation which attracted a good number of high-quality submissions. Results of this evaluation are published so that users of the data have external benchmarks against which to compare their novel approaches. |
| Type Of Material | Improvements to research infrastructure |
| Year Produced | 2023 |
| Provided To Others? | Yes |
| Impact | The tools and data were made publically available and researchers were invited to use it to compare their hearing aid intelligibility prediction algorithms. A number of teams from the US, Europe and Asia participated representing both academia and industry. Results were presented at a Workshop held in Dublin 2023 as a satellite event of the INTERSPEECH conference. |
| URL | https://claritychallenge.org/docs/cpc2/cpc2_intro |
| Title | Dataset of British English speech recordings for psychoacoustics and speech processing research |
| Description | The Clarity Speech Corpus is a forty speaker British English speech dataset. The corpus was created for the purpose of running listening tests to gauge speech intelligibility and quality in the Clarity Project, which has the goal of advancing speech signal processing by hearing aids through a series of challenges. The dataset is suitable for machine learning and other uses in speech and hearing technology, acoustics and psychoacoustics. The data comprises recordings of approximately 10,000 sentences drawn from the British National Corpus (BNC) with suitable length, words and grammatical construction for speech intelligibility testing. The collection process involved the selection of a subset of BNC sentences, the recording of these produced by 40 British English speakers, and the processing of these recordings to create individual sentence recordings with associated prompts and metadata.clarity_utterances.v1_2.tar.gz contains all the recordings as .wav files, with the accompanying metadata such as text prompts in clarity_master.json. Further details are given in the readme.Sample_clarity_utterances.zip contains a sample of 10.Please reference the following data paper, which has details on how the corpus was generated: Graetzer, S., Akeroyd, M.A., Barker, J., Cox, T.J., Culling, J.F., Naylor, G., Porter, E. and Muñoz, R.V., 2022. Dataset of British English speech recordings for psychoacoustics and speech processing research: the Clarity Speech Corpus. Data in Brief, p.107951. |
| Type Of Material | Database/Collection of data |
| Year Produced | 2022 |
| Provided To Others? | Yes |
| Impact | This dataset was used to enable the 1st Clarity Enhancement Challenge (CEC1) and will be used for further challenges within the project. CEC1 attracted participation from academic and industry teams from the US, Europe and Asia and resulted in a one day international workshop. |
| URL | https://salford.figshare.com/articles/dataset/Dataset_of_British_English_speech_recordings_for_psych... |
| Title | 1st Clarity Enhance Challenge Software |
| Description | Software to support the Clarity Enhancement Challenge. Python tools are provided for Mixing scenes to generate the training and development data Running the baseline hearing aid processing using the Oldenburg/Hortech open Master Hearing Aid (openMHA) software platform. Running signals through a python implementation of the Moore, Stone, Baer and Glasberg (MSBG) hearing loss model Estimating intelligibility using the Modified Binaural Short-Time Objective Intelligibility measure (MBSTOI). |
| Type Of Technology | Software |
| Year Produced | 2021 |
| Open Source License? | Yes |
| Impact | This software is being used by all participants in the 1st Clarity Enhancement Challenge, the first of the three rounds of machine learning for hearing aid processing challenges that this project is planning to deliver. The challenge and related software have only just been launched so it is a little early to assess impacts. However, it is already drawing interest internationally both from academic research groups and industry (e.g. Google). This broad participation is being directly enabled by the software we are making available. In particular, for groups who have not previously worked in the area of hearing impairment, the baseline hearing aid processor, hearing loss model and intelligibility prediction software are likely to be essential tools that they will not have previously had access to. |
| URL | https://zenodo.org/record/4593857 |
| Title | NOTE - Nottingham Online Testing Environment |
| Description | NOTE is an online platform for running audio experiments and questionnaires.Its main aims are [A] For audio, to present sounds in a reliable way (and to know just what code is doing the presenting) [B] For questionnaires, allow for various forms of responses including multiple choice and free text, etc. [C] Require active consent before proceeding [D] Tag data with participant codes and keeps all data secure.[E] Be open to participants when told the address, but passed-protected for data access and generally hidden from Google [F] Have a 'live' area of current experiments and a 'development' area for developing & debugging [G] to be stable on all modern browsers and on computers, tablets and phones. This award helped fund its ongoing development. |
| Type Of Technology | Webtool/Application |
| Year Produced | 2021 |
| Impact | When finished we intend to make it available to other researchers and we are hopeful it will prove influential. |
| URL | http://surveys.hearing.nottingham.ac.uk/notedemo1 |
| Title | claritychallenge/clarity_CC: clarity_CC release v1.2 |
| Description | Software to support the Clarity Enhancement and Prediction Challenges. Python tools are provided for • Mixing scenes to generate the training and development data • Running the baseline hearing aid processing using the Oldenburg/Hoertech open Master Hearing Aid (openMHA) software platform • Running signals through a python implementation of the Moore, Stone, Baer and Glasberg (MSBG) hearing loss model • Estimating intelligibility using the Modified Binaural Short-Time Objective Intelligibility measure (MBSTOI). This version includes • The first release of the code for supporting the recently opened 1st Clarity Prediction Challenge (CPC1) • Code for supporting the (now closed) 1st Clarity Enhancement Challenge (CEC1) |
| Type Of Technology | Software |
| Year Produced | 2021 |
| Impact | ... |
| URL | https://zenodo.org/record/4593856 |
| Title | pyclarity |
| Description | pyclarity is a software suite for machine learning challenges to enhance hearing-aid signal processing and to better predict how people perceive speech-in-noise (Clarity) and speech-in-music (Cadenza). Files can be accessed via the Related Materials links below. |
| Type Of Technology | Software |
| Year Produced | 2023 |
| Impact | Software to support the Clarity Enhancement Challenge. Python tools are provided for Mixing scenes to generate the training and development data Running the baseline hearing aid processing using the Oldenburg/Hortech open Master Hearing Aid (openMHA) software platform. Running signals through a python implementation of the Moore, Stone, Baer and Glasberg (MSBG) hearing loss model Estimating intelligibility using the Modified Binaural Short-Time Objective Intelligibility measure (MBSTOI). |
| URL | https://figshare.shef.ac.uk/articles/software/pyclarity/23230694 |
| Description | The 4th Clarity Workshop on Machine Learning Challenges for Hearing Aids (Clarity-2023) |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Other audiences |
| Results and Impact | The 4th Clarity Workshop on Machine Learning Challenges for Hearing Aids (Clarity-2023) was an in-person Workshop held in Dublin on 19th August 2023 and organised by the Clarity project team. It was supported by ISCA (International Speech Communication Association) and held as an official satellite event of the main annual ISCA conference, Interspeech. The workshop had attendees from the US, Europe and Asia and from both academia and industry. The event features invited talks from academia and industry (Meta Reality Labs, Sonova AG); and from a hearing aid user with audiological expertise providing insights regarding unsolved problems from a user perspective. The purpose of the event was to disseminate our project's findings arising from the 2nd Clarity Prediction Challenge; to foster discussion between academia and industry on the topic of the potential for machine learning in the hearing aid industry; and as an opportunity to get input from industry and academia concerning the directions for future datasets and challenges being created by the project. The input from the discussions has contributed to the design of the current round of data collection and the new design for speech intelligibility enhancement evaluation that we will be launching in 2024 as the 3rd Clarity Enhancement Challenge. |
| Year(s) Of Engagement Activity | 2023 |
| URL | https://claritychallenge.org/clarity2023-workshop/ |