📣 Help Shape the Future of UKRI's Gateway to Research (GtR)

We're improving UKRI's Gateway to Research and are seeking your input! If you would be interested in being interviewed about the improvements we're making and to have your say about how we can make GtR more user-friendly, impactful, and effective for the Research and Innovation community, please email gateway@ukri.org.

Estimating severity from multiple data sources using Bayesian evidence synthesis

Lead Research Organisation: University of Cambridge
Department Name: UNLISTED

Abstract

As we prepare for a possible COVID-19 pandemic, understanding the severity of the
epidemic, i.e. the proportion of infections that result in a severe event such as
hospitalisation or death, is crucial to monitoring and predicting the burden of the epidemic
on healthcare services. This burden is measured by the number of people infected who
require primary care from general practitioners, hospital admission, respiratory support
and/or admission to intensive care. Severity is quantified by infection-severity and caseseverity
risks, namely the probability that an infection (whether with or without symptoms)
or a symptomatic infection (clinical case) leads to a severe event. These quantities are
challenging to observe directly from a single dataset, as it is not possible to detect and
follow-up every case in a population, particularly early in the epidemic when case counts
may miss many asymptomatic or mild cases. The problem is compounded by the fact that
for patients still ill in hospital, we have not yet had time to observe whether they will
recover or not. We propose combining information from multiple datasets, both on
individuals and aggregated counts, to estimate severity while accounting for the
challenges of missing cases and not yet observing outcomes.

Technical Summary

This COVID-19 Rapid Response award is jointly funded (50:50) between the Medical Research Council and the National Institute for Health Research. The figure displayed is the total award amount of the two funders combined, with each partner contributing equally towards the project.

In preparing for a possible COVID-19 pandemic, estimates of severity, in particular of both
the numbers of infections occurring at different levels of severity and the infection- and
case-severity risks, are crucial to understand and predict the burden and impact of the
epidemic on healthcare services. Such estimates are most importantly needed by age and
risk group (e.g. defined by co-morbidities) strata, although in the early stages of an
epidemic, strata-specific information is rarely available. No single dataset can provide
enough information on its own to estimate severity, but estimation is feasible by
synthesising multiple datasets, such as: line-listing data from first few hundred type
studies; surveillance data including case counts, numbers accessing healthcare, and
numbers of deaths; cohort studies; and household studies. Such a synthesis needs to
account for biases inherent in each data source, including differential ascertainment by
severity level; and to account for the incomplete nature of the data, which, collected in real
time, are typically affected by censoring of final outcomes (recovery/hospital discharge or
death). We propose to make the best use of both individual- and aggregate-level data that
will become available, by using a combination of survival analysis techniques (e.g. curerate
mixture or competing risks models) and Bayesian evidence synthesis in a single
analysis to estimate severity in real time, as data accumulate over the course of the
epidemic, and once the epidemic is over.

Publications

10 25 50
 
Description Attendance at two meetings of SPI-M, modelling sub-group of SAGE
Geographic Reach National 
Policy Influence Type Participation in a guidance/advisory committee
URL https://www.gov.uk/government/publications/scientific-advisory-group-for-emergencies-sage-coronaviru...
 
Description National and International government advisory groups
Geographic Reach Europe 
Policy Influence Type Contribution to a national consultation/review
Impact Severity estimates for COVID-19, including probability of ICU admission given hospital admission, probabilities of death given hospital admission (with and without ICU admission), probabilities of discharge given hospital admission, lengths of stay in hospital and ICU, and relative severity (risks of hospitalisation and death) by variant, have regularly been provided to Public Health England's (PHE)/UKHSA Joint Modelling Team, Variant Technical Group and to the SPI-MO and SAGE advisory groups, providing evidence to inform public health decisions.
URL https://www.gov.uk/government/publications/investigation-of-sars-cov-2-variants-technical-briefings
 
Description Regione Lombardia Welfare Directorate
Geographic Reach Local/Municipal/Regional 
Policy Influence Type Implementation circular/rapid advice/letter to e.g. Ministry of Health
Impact A technical report describing estimates of COVID-19 severe burden in the Italian region of Lombardy (risks of competing outcomes in hospital and the community, including hospital admission, discharge/recovery, ICU admission and death, lengths of stay in hospital and ICU, times to recovery in the community) was provided to public health officials at the Welfare Directorate of Regione Lombardia, to inform their resource planning for current and future waves of COVID-19.
 
Description WHO Statistical Expert Working Groups, Global Influenza Programme & Pandemic Inflenza Severity Assessment: thresholding & rightsizing surveillance
Geographic Reach Multiple continents/international 
Policy Influence Type Participation in a guidance/advisory committee
Impact Updated WHO PISA guidelines, due to be published later in 2024; updated guidelines on sample size calculations for rightsizing surveillance systems, likely to be published by WHO as an appendix to guidelines on integrated surveillance later in 2024. Update March 2025: updated guidelines published: PISA guidelines at https://www.who.int/publications/i/item/9789240093881 and integrated surveillance guidance at https://www.who.int/publications/i/item/9789240101432
URL https://www.who.int/teams/global-influenza-programme
 
Description Agreement for Performance of Work - Emergency - COVID-19 severity assessment in WHO Europe member states, evidence synthesis
Amount $17,595 (USD)
Funding ID 2022/1201685-0 
Organisation World Health Organization Regional Office for Europe 
Sector Public
Country Denmark
Start 01/2022 
End 04/2022
 
Title Estimation of relative risks of hospital admission and death by COVID-19 variant 
Description Survival modelling (stratified Cox proportional hazards) to estimate the relative risks of severe events among COVID-19 cases by variant, e.g. Alpha vs pandemic original strain; Delta vs Alpha; Delta AY4.2 vs other Delta; Omicron BA.1 vs Delta. Model and code for model shared with UKHSA collaborators as knowledge transfer. Protocol and reuseable code for the model being drafted for sharing with WHO Europe member states to carry out similar analyses. Update March 2023: Protocol and preprint for standardised analyses across WHO Europe member states available here: https://arxiv.org/abs/2303.05541 and the accompanying code is publicy available here: https://github.com/TommyNyberg/variant_severity Update March 2024: Protocol and pilot for standardised analyses across WHO Europe member states now published (https://doi.org/10.2807/1560-7917.ES.2023.28.36.2300048) 
Type Of Material Data analysis technique 
Year Produced 2021 
Provided To Others? Yes  
Impact Presentation of resulting estimates to national and international government advisory groups. Four papers published or in press. Update March 2023: Standardised protocol, models and code used by several WHO Europe member states to analyse relative severity in their countries; and estimates pooled across Europe, as reported in the preprint linked above. 
URL https://arxiv.org/abs/2303.05541
 
Description BSU/UKHSA/ICNARC/Manchester/Liverpool/UCL/NHS Hospital Modelling 
Organisation Intensive Care National Audit and Research Centre
Country United Kingdom 
Sector Charity/Non Profit 
PI Contribution This partnership started as the Joint Modelling Team's Hospital Surge Working Group coordinated by UKHSA during the pandemic, to which we contributed statistical expertise and analyses of within-hospital severity of SARS-CoV-2. We are now applying for further grant funding to continue the partnership and widen the scope to winter pressures modelling.
Collaborator Contribution Mathematical modelling, epidemiology, emergency response, frontline clinical experience and expertise.
Impact Some of the papers attributed to this award arose through this multi-disciplinary collaboration (statisticians, epidemiologists, clinicians, mathematical modellers, operational researchers), the results of which went to the SAGE and SPI-M advisory groups during the pandemic (2020-2022); applications for further funding are underway.
Start Year 2020
 
Description BSU/UKHSA/ICNARC/Manchester/Liverpool/UCL/NHS Hospital Modelling 
Organisation National Health Service
Country United Kingdom 
Sector Hospitals 
PI Contribution This partnership started as the Joint Modelling Team's Hospital Surge Working Group coordinated by UKHSA during the pandemic, to which we contributed statistical expertise and analyses of within-hospital severity of SARS-CoV-2. We are now applying for further grant funding to continue the partnership and widen the scope to winter pressures modelling.
Collaborator Contribution Mathematical modelling, epidemiology, emergency response, frontline clinical experience and expertise.
Impact Some of the papers attributed to this award arose through this multi-disciplinary collaboration (statisticians, epidemiologists, clinicians, mathematical modellers, operational researchers), the results of which went to the SAGE and SPI-M advisory groups during the pandemic (2020-2022); applications for further funding are underway.
Start Year 2020
 
Description BSU/UKHSA/ICNARC/Manchester/Liverpool/UCL/NHS Hospital Modelling 
Organisation University College London
Country United Kingdom 
Sector Academic/University 
PI Contribution This partnership started as the Joint Modelling Team's Hospital Surge Working Group coordinated by UKHSA during the pandemic, to which we contributed statistical expertise and analyses of within-hospital severity of SARS-CoV-2. We are now applying for further grant funding to continue the partnership and widen the scope to winter pressures modelling.
Collaborator Contribution Mathematical modelling, epidemiology, emergency response, frontline clinical experience and expertise.
Impact Some of the papers attributed to this award arose through this multi-disciplinary collaboration (statisticians, epidemiologists, clinicians, mathematical modellers, operational researchers), the results of which went to the SAGE and SPI-M advisory groups during the pandemic (2020-2022); applications for further funding are underway.
Start Year 2020
 
Description BSU/UKHSA/ICNARC/Manchester/Liverpool/UCL/NHS Hospital Modelling 
Organisation University of Liverpool
Country United Kingdom 
Sector Academic/University 
PI Contribution This partnership started as the Joint Modelling Team's Hospital Surge Working Group coordinated by UKHSA during the pandemic, to which we contributed statistical expertise and analyses of within-hospital severity of SARS-CoV-2. We are now applying for further grant funding to continue the partnership and widen the scope to winter pressures modelling.
Collaborator Contribution Mathematical modelling, epidemiology, emergency response, frontline clinical experience and expertise.
Impact Some of the papers attributed to this award arose through this multi-disciplinary collaboration (statisticians, epidemiologists, clinicians, mathematical modellers, operational researchers), the results of which went to the SAGE and SPI-M advisory groups during the pandemic (2020-2022); applications for further funding are underway.
Start Year 2020
 
Description BSU/UKHSA/ICNARC/Manchester/Liverpool/UCL/NHS Hospital Modelling 
Organisation University of Manchester
Department School of Mathematics
Country United Kingdom 
Sector Academic/University 
PI Contribution This partnership started as the Joint Modelling Team's Hospital Surge Working Group coordinated by UKHSA during the pandemic, to which we contributed statistical expertise and analyses of within-hospital severity of SARS-CoV-2. We are now applying for further grant funding to continue the partnership and widen the scope to winter pressures modelling.
Collaborator Contribution Mathematical modelling, epidemiology, emergency response, frontline clinical experience and expertise.
Impact Some of the papers attributed to this award arose through this multi-disciplinary collaboration (statisticians, epidemiologists, clinicians, mathematical modellers, operational researchers), the results of which went to the SAGE and SPI-M advisory groups during the pandemic (2020-2022); applications for further funding are underway.
Start Year 2020
 
Description Cambridge / Imperial / UKHSA collaboration 
Organisation Imperial College London
Country United Kingdom 
Sector Academic/University 
PI Contribution Statistical expertise; coding; paper writing
Collaborator Contribution Data provision; epidemiological expertise; coding
Impact Paper on relative severity of Omicron variant compared to Delta, in press in The Lancet (publication date 17th March 2022). Update Feb 2024: paper published, collaboration not continued.
Start Year 2021
 
Description Cambridge / Imperial / UKHSA collaboration 
Organisation National Institute for Health and Care Research
Department Health Protection Research Unit for Modelling Methodology
Country United Kingdom 
Sector Academic/University 
PI Contribution Statistical expertise; coding; paper writing
Collaborator Contribution Data provision; epidemiological expertise; coding
Impact Paper on relative severity of Omicron variant compared to Delta, in press in The Lancet (publication date 17th March 2022). Update Feb 2024: paper published, collaboration not continued.
Start Year 2021
 
Description Cambridge / Imperial / UKHSA collaboration 
Organisation Public Health England
Department Centre of Infectious Disease Surveillance and Control
Country United Kingdom 
Sector Public 
PI Contribution Statistical expertise; coding; paper writing
Collaborator Contribution Data provision; epidemiological expertise; coding
Impact Paper on relative severity of Omicron variant compared to Delta, in press in The Lancet (publication date 17th March 2022). Update Feb 2024: paper published, collaboration not continued.
Start Year 2021
 
Description Harvard/Hong Kong collaboration 
Organisation Harvard University
Department Harvard T.H. Chan School of Public Health
Country United States 
Sector Academic/University 
PI Contribution Statistical expertise and analysis, particularly in Bayesian evidence synthesis for severity estimation
Collaborator Contribution Mathematical modelling and epidemiological expertise, particularly in infectious disease severity
Impact Prior to this grant, this multi-disciplinary collaboration (statistics, mathematical modelling, infectious disease epidemiology) resulted in several publications on influenza severity. Both partners are co-investigators on this grant. Update Feb 2024: Following the end of the severity grant, the collaboration with Hong Kong University has continued, leading to two exchange research visits, two drafted papers close to submission, and inclusion of AP as collaborator on a grant application to the Hong Kong General Research Fund.
Start Year 2009
 
Description Harvard/Hong Kong collaboration 
Organisation University of Hong Kong
Country Hong Kong 
Sector Academic/University 
PI Contribution Statistical expertise and analysis, particularly in Bayesian evidence synthesis for severity estimation
Collaborator Contribution Mathematical modelling and epidemiological expertise, particularly in infectious disease severity
Impact Prior to this grant, this multi-disciplinary collaboration (statistics, mathematical modelling, infectious disease epidemiology) resulted in several publications on influenza severity. Both partners are co-investigators on this grant. Update Feb 2024: Following the end of the severity grant, the collaboration with Hong Kong University has continued, leading to two exchange research visits, two drafted papers close to submission, and inclusion of AP as collaborator on a grant application to the Hong Kong General Research Fund.
Start Year 2009
 
Description JUNIPER (Joint UNIversities Pandemic and Epidemiological Research) Consortium 
Organisation United Kingdom Research and Innovation
Country United Kingdom 
Sector Public 
PI Contribution Contributed statistical input; estimates of severity.
Collaborator Contribution The JUNIPER Consortium, funded by UKRI-MRC (MR/V038613/1) consists of teams and research groups (Bristol, Cambridge, Exeter, Lancaster, Manchester, Oxford and Warwick) involved in the current effort to generate forecasts of the pandemic to inform the SPI-M and SAGE advisory groups, and the funding will support these teams in their continued forecasting and prediction roles. The consortium aims to build national capacity, train the next generation of epidemic modellers, and develop their modelling capacity. Work has until now focussed on hotspot detection and schools/universities.
Impact A paper on epidemic phase bias has been re-submitted following revision for Statistical Methods in Medical Research.
Start Year 2021
 
Description PHE/UKHSA COVID-19 collaboration 
Organisation Public Health England
Country United Kingdom 
Sector Public 
PI Contribution Statistical expertise & analysis, as a member of the PHE/UKHSA Joint Modelling Team, one of their cells in the national response to the COVID-19 pandemic.
Collaborator Contribution Provision of data, statistical and mathematical modelling expertise, one senior member of the PHE/UKHSA Joint Modelling Team is a co-Investigator on this grant, epidemiological and public health surveillance expertise via the PHE/UKHSA Epidemiology and Surveillance Cells of the COVID-19 response.
Impact Three papers have resulted from this multi-disciplinary collaboration, two are under revision for re-submission to Statistical Methods in Medical Research, one has been submitted to the BMJ. Crucially, results from this collaboration on the severity of COVID-19 have been regularly discussed at the PHE Joint Modelling Team meetings, and sent to the governernment advisory groups SPI-M (Scientific Pandemic Influenza Modelling) and SAGE (Scientific Advisory Group for Emergencies) as evidence informing the government response. Several co-Investigators on this grant are members of SPI-M. 2022 update: several further papers have resulted assessing the relative severity of different COVID-19 variants, presented regularly to the UKHSA Variant Technical Group, reported in UKHSA Variant Technical Briefings, and sent to SPI-M. Update Feb 2024: this collaboration has led to applications to the NIHR for further funding to continue the collaboration, working on integrating epidemiology, statistical/mathematical modelling and operational response for understanding the impacts of surges in respiratory infections on hospitals, patients and staff.
Start Year 2020
 
Description WHO Europe / ECDC severity protocol 
Organisation European Centre for Disease Prevention and Control (ECDC)
Country Sweden 
Sector Public 
PI Contribution Led a cross-Europe collaboration to develop, pilot and synthesise the results of a standard protocol for estimating the relative severity of different SARS-CoV-2 variants. Provided statistical expertise on the protocol and meta-analysis of the resulting country-specific results to obtain a Europe-wide result.
Collaborator Contribution Inputted surveillance/epidemiology expertise to the development of the protocol, carried out the country-specific analyses.
Impact A paper describing the protocol, analyses and results: https://doi.org/10.2807/1560-7917.ES.2023.28.36.2300048.
Start Year 2022
 
Description WHO Europe / ECDC severity protocol 
Organisation World Health Organization (WHO)
Department Regional Office for Europe (WHO/Europe)
Country Global 
Sector Public 
PI Contribution Led a cross-Europe collaboration to develop, pilot and synthesise the results of a standard protocol for estimating the relative severity of different SARS-CoV-2 variants. Provided statistical expertise on the protocol and meta-analysis of the resulting country-specific results to obtain a Europe-wide result.
Collaborator Contribution Inputted surveillance/epidemiology expertise to the development of the protocol, carried out the country-specific analyses.
Impact A paper describing the protocol, analyses and results: https://doi.org/10.2807/1560-7917.ES.2023.28.36.2300048.
Start Year 2022
 
Title R code for article published in Eurosurveillance 
Description R code for investigators in separate countries to estimate SARS-CoV-2 variant severity according to a standardised protocol and statistical analysis plan. Supplement to the article: Nyberg T, Bager P, Bech Svalgaard I, Bejko D, Bundle N, Evans J, Grove Krause T, McMenamin J, Mossong J, Mutch H, Omokanye A, Peralta-Santos A, Pinto-Leite P, Starrfelt J, Thelwall S, Veneti L, Whittaker R, Wood J, Pebody R, Presanis AM. A standardised protocol for relative SARS-CoV-2 variant severity assessment, applied to Omicron BA.1 and Delta in six European countries, October 2021 to February 2022. Eurosurveillance 2023 28(36): pii=2300048. 
Type Of Technology Software 
Year Produced 2023 
Open Source License? Yes  
Impact The R code was used by national public health agencies in six European countries to estimate the relative severity of the SARS-CoV-2 variants Omicron BA.1 and Delta based on their national COVID-19 surveillance data, as described in the publication: Nyberg T, Bager P, Bech Svalgaard I, Bejko D, Bundle N, Evans J, Grove Krause T, McMenamin J, Mossong J, Mutch H, Omokanye A, Peralta-Santos A, Pinto-Leite P, Starrfelt J, Thelwall S, Veneti L, Whittaker R, Wood J, Pebody R, Presanis AM. A standardised protocol for relative SARS-CoV-2 variant severity assessment, applied to Omicron BA.1 and Delta in six European countries, October 2021 to February 2022. Eurosurveillance 2023 28(36): pii=2300048. 
URL https://github.com/TommyNyberg/variant_severity
 
Title R package for flexible survival and multi-state modelling: flexsurv 
Description Software developed for flexible survival and multi-state modelling by my colleague at MRC Biostatistics Unit, University of Cambridge, Christopher Jackson. It was developed prior to this award, but was signifcantly expanded during this award, motivated by the problem of estimating COVID-19 severity, among hospitalised cases. 
Type Of Technology Software 
Year Produced 2021 
Open Source License? Yes  
Impact Two papers published (Grosso et al 2021, Presanis et al 2021) and two in press (Jackson et al 2022a, 2022b) using the software package to estimate risks of competing outcomes among hospitalised COVID-19 cases (ICU admission, death, discharge) and lengths of stay in hospital. 
URL https://cran.r-project.org/package=flexsurv
 
Title pkirwan/COVID-hospital-outcomes: R code for manuscript 
Description Trends in COVID-19 hospital outcomes in England before and after vaccine introduction, a cohort study 
Type Of Technology Software 
Year Produced 2022 
Open Source License? Yes  
Impact This code accompanies a paper on estimating hospital-fatality risks and lengths of stay among COVID-19 patients. Initial results were presented to the Scientific Advisory Group for Emergencies in December 2021: https://www.gov.uk/government/publications/mrc-biostatistics-unit-and-phe-estimates-of-covid-19-hospitalised-mortality-and-length-of-stay-data-from-march-2020-to-september-2021-7-december-20 
URL https://zenodo.org/record/6856530
 
Description Invited speaker - WHO Europe and ECDC joint surveillance meeting (respiratory infections) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Presentation of our research on estimating COVID-19 severity by variant to the member states of WHO Europe and ECDC. Let to emergency funding for us to work with these member states to support them in producing similar analyses, both by writing a protocol for the analysis and sharing code/methods/support, and to synthesise the resulting estimates across the involved countries to produce more precise estimates than from one country alone.
Year(s) Of Engagement Activity 2021
 
Description Invited speaker at Joint WHO Europe/ECDC surveillance meeting 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Invited talk at the Joint WHO Regional Office for Europe & European Centre for Disease Prevention and Control Annual European Influenza and COVID-19 Surveillance Meeting 2023, Copenhagen/virtual, 31 May-2 Jun 2023, on the topic of an international collaboration to assess SARS-CoV-2 variant infection case-severity in Europe.
Year(s) Of Engagement Activity 2023
 
Description Presentation on COVID-19 severity estimation to CMStatistics Conference 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Presented our work of the last 18 months of the pandemic on estimating its severity to other statistical and epidemiological colleagues, prompting questions and discussion.
Year(s) Of Engagement Activity 2021
URL http://www.cmstatistics.org/CMStatistics2021/programme.php
 
Description Presentation on trends in COVID-19 hospital outcomes at UKHSA conference 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Presentation at UKHSA conference entitled: Trends in COVID-19 hospital outcomes in England before and after vaccine introduction, 2020-2021: a cohort study. Presentation was made to other statistical and epidemiological colleagues, prompting questions and discussion.
Year(s) Of Engagement Activity 2022
 
Description Press coverage on paper showing Delta variant of COVID-19 caused more hospitalisations than Alpha variant 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Press coverage on paper showing Delta variant of COVID-19 caused more hospitalisations than Alpha variant. Relates to paper published in The Lancet Infectious Diseases. Research was covered in 644 national, international and regional media outlets.
Year(s) Of Engagement Activity 2021