RELOAD: REspiratory disease progression through LOngitudinal Audio Data machine learning
Lead Research Organisation:
University of Cambridge
Department Name: Computer Science and Technology
Abstract
Respiratory Tract Infections (RTIs) are the most common cause of illness. This was true even before the COVID-19 pandemic. They are most often the reason patients consult a GP. The illness they cause is usually mild, but in some cases can become severe, and occasionally can lead to death. Around half of all antibiotic prescriptions are for RTIs. Most people with an RTI get better without needing treatment. However, we need to notice quickly when people are getting seriously ill. If we do not, the effect on them and on healthcare services can be large. Doctors have rules and tests that help them identify patients who are more likely to need treatment, but these do not work well for every patient. Also, they are not useful for helping patients manage their own illness. Using machine learning (AI systems) to analyse breathing and speech sounds automatically could be a game-changer. Firstly, it could reassure many patients that they do not need to see a doctor. Secondly, it could reduce prescriptions for antibiotics by identifying patients who will get better on their own. Identifying patients at higher risk could also reduce hospital admissions, cases of severe illness and the number who die. All these effects would reduce pressure on the NHS. We already know that some signs, such as breathing faster, can tell us whether an RTI is getting worse, and we know we can measure these signs by recording the sound of the breath. We know that RTIs also affect breathing pattern, the sound of speech and trying to breathe when speaking. We believe that other breathing sounds and patterns are also likely to change when you get an RTI and this is something we want to explore in this project. We aim to find information in sound recordings of breathing, cough and speech which changes in a way we can predict as a person gets sicker or recovers. We will need to research the sounds we should record and how we should analyse them to get the most useful information. A study into how these sounds change over time will give us added information, not previously explored in any great depth. We have already worked with sounds from people with COVID-19, so we know lots of people will volunteer to take part and give us their sound data if we give them an app. We know this is a very cost-effective way to study how symptoms of a disease change over time.
To be confident about using a machine learning system to treat patients, Doctors need to know if it is giving good advice. If they know a sound recording or a prediction is not very dependable, they can make sure they do extra checks or ask the patient to re-record their sounds. We plan to develop a machine learning system that can rate how reliable its own advice is each time. This will help doctors to know when to trust the system. Designing machine learning systems that can tell us about the quality of their advice is something new we will be exploring in this study.
Our project will ask volunteers to use an app to collect speech and breathing sound data. They will be asked to make a recording when they are healthy and then another one every day if they get an RTI. The app will also collect other health information from them, such as any medication they take and any other illness they may have. The machine learning system will process the data to predict whether they are getting better or worse and rate its own confidence in its prediction. GPs will use patients' medical records to tell us which of the volunteers comes to see their doctor for treatment and whether anyone had to go to hospital. This will allow us to assess the quality of the advice from the machine learning system. Our aim is to develop a machine learning system that can assess if someone with an RTI should see their doctor for advice or can expect to get better without treatment.
To be confident about using a machine learning system to treat patients, Doctors need to know if it is giving good advice. If they know a sound recording or a prediction is not very dependable, they can make sure they do extra checks or ask the patient to re-record their sounds. We plan to develop a machine learning system that can rate how reliable its own advice is each time. This will help doctors to know when to trust the system. Designing machine learning systems that can tell us about the quality of their advice is something new we will be exploring in this study.
Our project will ask volunteers to use an app to collect speech and breathing sound data. They will be asked to make a recording when they are healthy and then another one every day if they get an RTI. The app will also collect other health information from them, such as any medication they take and any other illness they may have. The machine learning system will process the data to predict whether they are getting better or worse and rate its own confidence in its prediction. GPs will use patients' medical records to tell us which of the volunteers comes to see their doctor for treatment and whether anyone had to go to hospital. This will allow us to assess the quality of the advice from the machine learning system. Our aim is to develop a machine learning system that can assess if someone with an RTI should see their doctor for advice or can expect to get better without treatment.
Publications
| Description | The project released an app to collect data from participants about their respiratory infection audio signals. GP practices were recruited to advertise the app with patients. about 500 users were recruited and recorded sounds. we are in the process of analysing the data however we have already applied some machine learning analysis to it and we have initial promising results on some of the predictions we are targeting. we have also published an open foundation model for respiratory health which is a game changer in the way we work with small datasets of this kind. We have organised a first of its kind workshop bringing together experts on this topic from academia and industry. |
| Exploitation Route | the app developed is of great value for follow on data collections of related respiratory diseases the model developed is one which others can use for their respiratory audio tasks |
| Sectors | Digital/Communication/Information Technologies (including Software) |
| Description | we have released a model which can be used by all to improve their respiratory health machine learning task performance |
| First Year Of Impact | 2024 |
| Sector | Healthcare |
| Title | Machine learning model |
| Description | OPERA system allows us to: Curate a unique large-scale(~136K samples, 400+ hours), multi-source (5 datasets), multi-modal (breathing, coughing, and lung sounds) and publicly available (or under controlled access) dataset for model pretraining. Pretrain 3 generalizable acoustic models with the curated unlabeled data using contrastive learning and generative pretraining, and release the model checkpoints. Employ 10 labeled datasets (6 not covered by pretraining) to formulate 19 respiratory health tasks, ensuring fair, comprehensive and reproducible downstream evaluation. Enable researchers and developers to extract feature using our model, or develop new models with our data and system, as a starting point for future exploration. |
| Type Of Material | Technology assay or reagent |
| Year Produced | 2024 |
| Provided To Others? | Yes |
| Impact | OPERA system allows us to: Curate a unique large-scale(~136K samples, 400+ hours), multi-source (5 datasets), multi-modal (breathing, coughing, and lung sounds) and publicly available (or under controlled access) dataset for model pretraining. Pretrain 3 generalizable acoustic models with the curated unlabeled data using contrastive learning and generative pretraining, and release the model checkpoints. Employ 10 labeled datasets (6 not covered by pretraining) to formulate 19 respiratory health tasks, ensuring fair, comprehensive and reproducible downstream evaluation. Enable researchers and developers to extract feature using our model, or develop new models with our data and system, as a starting point for future exploration. |
| URL | https://opera-benchmark.github.io |
| Description | Nokia Bell Labs |
| Organisation | Nokia |
| Department | Nokia Bell Labs |
| Country | United States |
| Sector | Private |
| PI Contribution | Nokia Bell Labs is a project partner and we have been interacting with them machine learning for audio techniques. |
| Collaborator Contribution | They contributed a PhD scholarship |
| Impact | not yet |
| Start Year | 2023 |
| Description | Pfizer |
| Organisation | Pfizer Global R & D |
| Country | United States |
| Sector | Private |
| PI Contribution | we will contribute knowhow on audio based machine learning for health |
| Collaborator Contribution | the collaboration is not yet started but will pay for a phd studentship and allow data access |
| Impact | not yet |
| Start Year | 2024 |
| Title | mobile app |
| Description | the mobile app software is partly available open source |
| Type Of Technology | New/Improved Technique/Technology |
| Year Produced | 2024 |
| Impact | the app was recently released on the Apple App Store so that data collection can begin and we are currently testing the Android version. |
| URL | https://www.southampton.ac.uk/primarycare/reload.page |
| Description | Audio based AI for Respiratory Health Monitoring |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Postgraduate students |
| Results and Impact | A workshop bringing together top experts in academia, industry on Ai for respiratory health. |
| Year(s) Of Engagement Activity | 2025 |
| URL | https://www.eventbrite.co.uk/e/audio-based-ai-for-respiratory-health-monitoring-tickets-108387101036... |
| Description | PPIE event about use of smartphone microphone apps for respiratory health monitoring |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | Regional |
| Primary Audience | Public/other audiences |
| Results and Impact | event for the public on 27th Feb to share thoughts on using AI to improve healthcare. Discussion on our study and on the app that helps assess whether people with a cough are likely to get better or worse. Feedback gathered for shaping the future of healthcare technology. |
| Year(s) Of Engagement Activity | 2025 |
| URL | https://forms.office.com/Pages/ResponsePage.aspx?id=-XhTSvQpPk2-iWadA62p2AMG6TttB0FOsYPoNpBtsExUQ0sy... |