📣 Help Shape the Future of UKRI's Gateway to Research (GtR)

We're improving UKRI's Gateway to Research and are seeking your input! If you would be interested in being interviewed about the improvements we're making and to have your say about how we can make GtR more user-friendly, impactful, and effective for the Research and Innovation community, please email gateway@ukri.org.

RELOAD: REspiratory disease progression through LOngitudinal Audio Data machine learning

Lead Research Organisation: University of Cambridge
Department Name: Computer Science and Technology

Abstract

Respiratory Tract Infections (RTIs) are the most common cause of illness. This was true even before the COVID-19 pandemic. They are most often the reason patients consult a GP. The illness they cause is usually mild, but in some cases can become severe, and occasionally can lead to death. Around half of all antibiotic prescriptions are for RTIs. Most people with an RTI get better without needing treatment. However, we need to notice quickly when people are getting seriously ill. If we do not, the effect on them and on healthcare services can be large. Doctors have rules and tests that help them identify patients who are more likely to need treatment, but these do not work well for every patient. Also, they are not useful for helping patients manage their own illness. Using machine learning (AI systems) to analyse breathing and speech sounds automatically could be a game-changer. Firstly, it could reassure many patients that they do not need to see a doctor. Secondly, it could reduce prescriptions for antibiotics by identifying patients who will get better on their own. Identifying patients at higher risk could also reduce hospital admissions, cases of severe illness and the number who die. All these effects would reduce pressure on the NHS. We already know that some signs, such as breathing faster, can tell us whether an RTI is getting worse, and we know we can measure these signs by recording the sound of the breath. We know that RTIs also affect breathing pattern, the sound of speech and trying to breathe when speaking. We believe that other breathing sounds and patterns are also likely to change when you get an RTI and this is something we want to explore in this project. We aim to find information in sound recordings of breathing, cough and speech which changes in a way we can predict as a person gets sicker or recovers. We will need to research the sounds we should record and how we should analyse them to get the most useful information. A study into how these sounds change over time will give us added information, not previously explored in any great depth. We have already worked with sounds from people with COVID-19, so we know lots of people will volunteer to take part and give us their sound data if we give them an app. We know this is a very cost-effective way to study how symptoms of a disease change over time.
To be confident about using a machine learning system to treat patients, Doctors need to know if it is giving good advice. If they know a sound recording or a prediction is not very dependable, they can make sure they do extra checks or ask the patient to re-record their sounds. We plan to develop a machine learning system that can rate how reliable its own advice is each time. This will help doctors to know when to trust the system. Designing machine learning systems that can tell us about the quality of their advice is something new we will be exploring in this study.
Our project will ask volunteers to use an app to collect speech and breathing sound data. They will be asked to make a recording when they are healthy and then another one every day if they get an RTI. The app will also collect other health information from them, such as any medication they take and any other illness they may have. The machine learning system will process the data to predict whether they are getting better or worse and rate its own confidence in its prediction. GPs will use patients' medical records to tell us which of the volunteers comes to see their doctor for treatment and whether anyone had to go to hospital. This will allow us to assess the quality of the advice from the machine learning system. Our aim is to develop a machine learning system that can assess if someone with an RTI should see their doctor for advice or can expect to get better without treatment.

Publications

10 25 50
 
Description The project released an app to collect data from participants about their respiratory infection audio signals. GP practices were recruited to advertise the app with patients. about 500 users were recruited and recorded sounds. we are in the process of analysing the data however we have already applied some machine learning analysis to it and we have initial promising results on some of the predictions we are targeting.

we have also published an open foundation model for respiratory health which is a game changer in the way we work with small datasets of this kind.
We have organised a first of its kind workshop bringing together experts on this topic from academia and industry.
Exploitation Route the app developed is of great value for follow on data collections of related respiratory diseases
the model developed is one which others can use for their respiratory audio tasks
Sectors Digital/Communication/Information Technologies (including Software)

 
Description we have released a model which can be used by all to improve their respiratory health machine learning task performance
First Year Of Impact 2024
Sector Healthcare
 
Title Machine learning model 
Description OPERA system allows us to: Curate a unique large-scale(~136K samples, 400+ hours), multi-source (5 datasets), multi-modal (breathing, coughing, and lung sounds) and publicly available (or under controlled access) dataset for model pretraining. Pretrain 3 generalizable acoustic models with the curated unlabeled data using contrastive learning and generative pretraining, and release the model checkpoints. Employ 10 labeled datasets (6 not covered by pretraining) to formulate 19 respiratory health tasks, ensuring fair, comprehensive and reproducible downstream evaluation. Enable researchers and developers to extract feature using our model, or develop new models with our data and system, as a starting point for future exploration. 
Type Of Material Technology assay or reagent 
Year Produced 2024 
Provided To Others? Yes  
Impact OPERA system allows us to: Curate a unique large-scale(~136K samples, 400+ hours), multi-source (5 datasets), multi-modal (breathing, coughing, and lung sounds) and publicly available (or under controlled access) dataset for model pretraining. Pretrain 3 generalizable acoustic models with the curated unlabeled data using contrastive learning and generative pretraining, and release the model checkpoints. Employ 10 labeled datasets (6 not covered by pretraining) to formulate 19 respiratory health tasks, ensuring fair, comprehensive and reproducible downstream evaluation. Enable researchers and developers to extract feature using our model, or develop new models with our data and system, as a starting point for future exploration. 
URL https://opera-benchmark.github.io
 
Description Nokia Bell Labs 
Organisation Nokia
Department Nokia Bell Labs
Country United States 
Sector Private 
PI Contribution Nokia Bell Labs is a project partner and we have been interacting with them machine learning for audio techniques.
Collaborator Contribution They contributed a PhD scholarship
Impact not yet
Start Year 2023
 
Description Pfizer 
Organisation Pfizer Global R & D
Country United States 
Sector Private 
PI Contribution we will contribute knowhow on audio based machine learning for health
Collaborator Contribution the collaboration is not yet started but will pay for a phd studentship and allow data access
Impact not yet
Start Year 2024
 
Title mobile app 
Description the mobile app software is partly available open source 
Type Of Technology New/Improved Technique/Technology 
Year Produced 2024 
Impact the app was recently released on the Apple App Store so that data collection can begin and we are currently testing the Android version. 
URL https://www.southampton.ac.uk/primarycare/reload.page
 
Description Audio based AI for Respiratory Health Monitoring 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact A workshop bringing together top experts in academia, industry on Ai for respiratory health.
Year(s) Of Engagement Activity 2025
URL https://www.eventbrite.co.uk/e/audio-based-ai-for-respiratory-health-monitoring-tickets-108387101036...
 
Description PPIE event about use of smartphone microphone apps for respiratory health monitoring 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact event for the public on 27th Feb to share thoughts on using AI to improve healthcare. Discussion on our study and on the app that helps assess whether
people with a cough are likely to get better or worse. Feedback gathered for
shaping the future of healthcare technology.
Year(s) Of Engagement Activity 2025
URL https://forms.office.com/Pages/ResponsePage.aspx?id=-XhTSvQpPk2-iWadA62p2AMG6TttB0FOsYPoNpBtsExUQ0sy...