Functional relationships in Bayesian evidence synthesis of multiple outcomes
Lead Research Organisation:
University of Bristol
Department Name: Social Medicine
Abstract
Healthcare policy and clinical decisions are usually based on a systemic review, critical appraisal and statistical summary of the evidence from previous clinical trials. It is hoped that this will provide information on which treatment or treatments should be recommended, based on how effective they are on a particular measure (for example that patients' pain improves by a greater amount). However, for many clinical conditions several different tools are used to assess patients' response to treatment. Treatment effects on each of these tools (outcomes) can also be reported in different formats. This is often the case in clinical guidelines produced for the UK's National Institute for Health and Care Excellence (NICE), where different, but often related, outcomes are considered equally important to guide recommendations for clinical practice.
Performing separate analyses on each outcome, ignores the fact that outcomes are related and can lead to different or even conflicting conclusions. It is therefore desirable to summarise all the evidence on all outcomes in a coherent way. This makes best use of available evidence and will provide better (more precise) information on which to base clinical decisions and make policy recommendations. Current methods for summarising evidence on more than one outcome are technically difficult to implement and often do not greatly improve the precision of results. We propose more restrictive methods, which use expert opinion on how the various outcomes are related to obtain better estimates of treatment effects. For example, in the treatment of Social Anxiety, it is known that if patients' symptoms (measured on a particular scale) do not improve with treatment, then no patients will be classified as having recovered from the condition (measured on another scale). Conversely, treatments which improve symptoms are also more likely to have more patients achieving recovery. We propose methods for summarising the evidence on different, but related outcomes, by taking advantage of known relationships such as these, in order to obtain better treatment effect estimates.
The proposed methods are technically simpler to implement than the methods currently proposed for summarising multiple outcomes. However, by imposing relationship between the treatment effects on the different outcomes, we will be making strong assumptions about the data. These assumptions will be checked using recommended statistical techniques and expert clinical opinion, to ensure that the proposed models are appropriate. We will compare the proposed methods to the current standards for summarising treatment effects on multiple outcomes.
We propose to apply the methods to different clinical settings, develop methods to specify the relationships between outcomes and collaborate with clinical colleagues and decision makers to ensure that the assumptions are sensible and not contradicted by available data. We aim to develop a strategy which can be followed in order to obtain clinical opinion on plausible relationships between the outcomes, implement them in a statistical model and evaluate that model using available statistical software, to check that the assumptions made are appropriate.
Performing separate analyses on each outcome, ignores the fact that outcomes are related and can lead to different or even conflicting conclusions. It is therefore desirable to summarise all the evidence on all outcomes in a coherent way. This makes best use of available evidence and will provide better (more precise) information on which to base clinical decisions and make policy recommendations. Current methods for summarising evidence on more than one outcome are technically difficult to implement and often do not greatly improve the precision of results. We propose more restrictive methods, which use expert opinion on how the various outcomes are related to obtain better estimates of treatment effects. For example, in the treatment of Social Anxiety, it is known that if patients' symptoms (measured on a particular scale) do not improve with treatment, then no patients will be classified as having recovered from the condition (measured on another scale). Conversely, treatments which improve symptoms are also more likely to have more patients achieving recovery. We propose methods for summarising the evidence on different, but related outcomes, by taking advantage of known relationships such as these, in order to obtain better treatment effect estimates.
The proposed methods are technically simpler to implement than the methods currently proposed for summarising multiple outcomes. However, by imposing relationship between the treatment effects on the different outcomes, we will be making strong assumptions about the data. These assumptions will be checked using recommended statistical techniques and expert clinical opinion, to ensure that the proposed models are appropriate. We will compare the proposed methods to the current standards for summarising treatment effects on multiple outcomes.
We propose to apply the methods to different clinical settings, develop methods to specify the relationships between outcomes and collaborate with clinical colleagues and decision makers to ensure that the assumptions are sensible and not contradicted by available data. We aim to develop a strategy which can be followed in order to obtain clinical opinion on plausible relationships between the outcomes, implement them in a statistical model and evaluate that model using available statistical software, to check that the assumptions made are appropriate.
Technical Summary
Health Technology Assessments usually include systematic reviews of randomised controlled trials and evidence syntheses which are then embedded in economic models. While pair-wise meta-analysis has been extended to mixed treatment comparisons (MTC, also termed network meta-analysis) for multiple treatments, less progress has been made in synthesis of multiple outcomes, both within and between trials. A single, coherent analysis using all the available data on all outcomes is preferable to separate outcome-specific analyses. The commonly recommended approach, multivariate meta-analysis, requires only correlations between outcomes at the within- and between-trial levels, but models are technically difficult to fit, especially when there are more than 3 or 4 outcomes, and/or when the data are sparse. However, there is often prior knowledge on the nature of the relationships between outcomes and there is a need for models which capture these relationships. We will develop models that express functional relationships between outcomes, and which typically have fewer parameters. However, these models make stronger assumptions, so the issue of model fit is critical.
Trial data in different areas of medicine displays a wide variety of functional relationships between outcomes. We will focus on a very common occurring evidence structures that feature multiple outcomes subject to either measurement error (for example, patient- or clinician-rated outcomes) or variability (weight, blood pressure), reported in different formats and/ or at multiple time points.
We will develop a framework, applicable to all multiple outcome structures, for identifying, fitting, and evaluating functional relationships. We will apply the framework to several illustrative, real-life clinical examples. Models will be implemented by extending current Bayesian methods for MTC, of which standard (two-treatment) meta-analysis is a special case.
Trial data in different areas of medicine displays a wide variety of functional relationships between outcomes. We will focus on a very common occurring evidence structures that feature multiple outcomes subject to either measurement error (for example, patient- or clinician-rated outcomes) or variability (weight, blood pressure), reported in different formats and/ or at multiple time points.
We will develop a framework, applicable to all multiple outcome structures, for identifying, fitting, and evaluating functional relationships. We will apply the framework to several illustrative, real-life clinical examples. Models will be implemented by extending current Bayesian methods for MTC, of which standard (two-treatment) meta-analysis is a special case.
Planned Impact
Academic:
Outputs from this project will be published in leading journals, presented at international conferences and local meetings, and disseminated through the network of academic colleagues the applicant (PI) regularly collaborates with. Attendance at four international conferences is planned, some aimed at statistical audiences, others at health economics, decision-making and general evidence synthesis audiences. This will ensure that the research impacts on current knowledge in the fields of Bayesian evidence synthesis, multivariate meta-analysis and mixed treatment comparisons (network meta-analysis) for decision making, contributing to the advance of knowledge in these fields.
National Policy makers:
The PI is involved with the National Institute for Health and Care Excellence (NICE) Clinical Guidelines Technical Support Unit which, amongst other things, provides training and support to NICE guideline development groups on complex evidence synthesis. This involvement is expected to continue throughout the project and the methods will be immediately available for use in NICE guideline development, ensuring better use of the available evidence to make recommendations for clinical practice. Findings from this project will be incorporated into the training workshops currently provided to NICE, ensuring that the community of guideline developers and commissioners is aware of the potential benefits of the proposed approach, and can apply it when relevant.
International Policy makers and Industry:
The PI is involved with the delivery of 3- and 5-day courses on evidence synthesis, which are attended by researchers from the university departments contracted to undertake evaluations for NICE, by NICE analysts, and representatives of pharmaceutical companies, consultancy firms, policy makers and academic institutions from around the world. It is expected that work arising from this project will be incorporated into these courses, lectures and workshops, disseminating it to a wide audience and ensuring that it is used when relevant, impacting on policy and improving patient care worldwide.
Patients and their families:
Patients will benefit from methods that provide more accurate estimates of treatment benefits. The engagement with NICE, its partners and the pharmaceutical and consultancy industries, will allow the methods to be applied and understood by decision-making bodies and companies submitting evidence to them, ensuring more reliable information is available to make better decisions for patient care. This will result in a more efficient use of health resources, benefiting patients and society by improving health and well being.
Outputs from this project will be published in leading journals, presented at international conferences and local meetings, and disseminated through the network of academic colleagues the applicant (PI) regularly collaborates with. Attendance at four international conferences is planned, some aimed at statistical audiences, others at health economics, decision-making and general evidence synthesis audiences. This will ensure that the research impacts on current knowledge in the fields of Bayesian evidence synthesis, multivariate meta-analysis and mixed treatment comparisons (network meta-analysis) for decision making, contributing to the advance of knowledge in these fields.
National Policy makers:
The PI is involved with the National Institute for Health and Care Excellence (NICE) Clinical Guidelines Technical Support Unit which, amongst other things, provides training and support to NICE guideline development groups on complex evidence synthesis. This involvement is expected to continue throughout the project and the methods will be immediately available for use in NICE guideline development, ensuring better use of the available evidence to make recommendations for clinical practice. Findings from this project will be incorporated into the training workshops currently provided to NICE, ensuring that the community of guideline developers and commissioners is aware of the potential benefits of the proposed approach, and can apply it when relevant.
International Policy makers and Industry:
The PI is involved with the delivery of 3- and 5-day courses on evidence synthesis, which are attended by researchers from the university departments contracted to undertake evaluations for NICE, by NICE analysts, and representatives of pharmaceutical companies, consultancy firms, policy makers and academic institutions from around the world. It is expected that work arising from this project will be incorporated into these courses, lectures and workshops, disseminating it to a wide audience and ensuring that it is used when relevant, impacting on policy and improving patient care worldwide.
Patients and their families:
Patients will benefit from methods that provide more accurate estimates of treatment benefits. The engagement with NICE, its partners and the pharmaceutical and consultancy industries, will allow the methods to be applied and understood by decision-making bodies and companies submitting evidence to them, ensuring more reliable information is available to make better decisions for patient care. This will result in a more efficient use of health resources, benefiting patients and society by improving health and well being.
Organisations
People |
ORCID iD |
Sofia Dias (Principal Investigator) |
Publications
Medley N
(2015)
Outcomes reported in trials of methods for the induction of labour
in Trials
Nixon R
(2015)
Re: Hutchinson M, Fox RJ, Havrdova E, et al. Efficacy and safety of BG-12 (dimethyl fumarate) and other disease modifying therapies for the treatment of relapsing-remitting multiple sclerosis: a systematic review and mixed treatment comparison. Curr Med Res Opin 2014;30:613-27.
in Current medical research and opinion
Thom H
(2015)
The Cost-Effectiveness Of Novel Oral Anticoagulants For The Prevention Of Stroke In Atrial Fibrillation In England And Wales
in Value in Health
Schwingshackl L
(2015)
Dietary supplements and risk of cause-specific death, cardiovascular disease, and cancer: a protocol for a systematic review and network meta-analysis of primary prevention trials.
in Systematic reviews
Ades AE
(2015)
Simultaneous synthesis of treatment effects and mapping to a common scale: an alternative to standardisation.
in Research synthesis methods
Navarese EP
(2015)
Comparative efficacy and safety of anticoagulant strategies for acute coronary syndromes. Comprehensive network meta-analysis of 42 randomised trials involving 117,353 patients.
in Thrombosis and haemostasis
Turner NL
(2015)
A Bayesian framework to account for uncertainty due to missing binary outcome data in pairwise meta-analysis.
in Statistics in medicine
Bryden P
(2015)
A Cost-Effectiveness Analysis Of Novel Oral Anticoagulants For Acute Treatment And Secondary Prevention Of Venous Thromboembolic Disease
in Value in Health
Petrou P
(2016)
A mixed treatment comparison for short- and long-term outcomes of bare-metal and drug-eluting coronary stents.
in International journal of cardiology
Alfirevic Z
(2016)
Methods to induce labour: a systematic review, network meta-analysis and cost-effectiveness analysis.
in BJOG : an international journal of obstetrics and gynaecology
Remonti LR
(2016)
Classes of antihypertensive agents and mortality in hypertensive patients with type 2 diabetes-Network meta-analysis of randomized trials.
in Journal of diabetes and its complications
Van Valkenhoef G
(2016)
Automated generation of node-splitting models for assessment of inconsistency in network meta-analysis.
in Research synthesis methods
Pedder H
(2016)
Data extraction for complex meta-analysis (DECiMAL) guide
in Systematic Reviews
Alfirevic Z
(2016)
Which method is best for the induction of labour? A systematic review, network meta-analysis and cost-effectiveness analysis.
in Health technology assessment (Winchester, England)
Dias S
(2016)
Absolute or relative effects? Arm-based synthesis of trial data.
in Research synthesis methods
Mawdsley D
(2016)
Model-Based Network Meta-Analysis: A Framework for Evidence Synthesis of Clinical Trial Data.
in CPT: pharmacometrics & systems pharmacology
Keeney E
(2017)
Different Methods for Modelling Severe Hypoglycaemic Events: Implications for Effectiveness and Cost-Effectiveness Analyses
in Value in Health
Gajulapalli RD
(2017)
Optimal duration of dual antiplatelet therapy after drug eluting stent implantation: a network meta-analysis.
in Anatolian journal of cardiology
Sarri G
(2017)
Vasomotor symptoms resulting from natural menopause: a systematic review and network meta-analysis of treatment effects from the National Institute for Health and Care Excellence guideline on menopause
in BJOG: An International Journal of Obstetrics & Gynaecology
Donegan S
(2017)
Network meta-analysis including treatment by covariate interactions: Consistency can vary across covariate values.
in Research synthesis methods
Sarri G
(2017)
Vasomotor Symptoms Resulting From Natural Menopause: A Systematic Review and Network Meta-analysis of Treatment Effects From the National Institute for Health and Care Excellence Guideline on Menopause
in Obstetrical & Gynecological Survey
Oba Y
(2017)
Fixed-dose combination inhalers compared to long-acting bronchodilators for COPD: a network meta-analysis
in Cochrane Database of Systematic Reviews
Guideline Title | 8-year surveillance of Induction of labour NICE guideline CG170 |
Description | NICE guideline update |
Geographic Reach | National |
Policy Influence Type | Citation in clinical guidelines |
Impact | The work described in our publication meant that NICE are updating the guideline with new evidence which will improve healthcare of pregnant women and their babies, as well as ensuring the most cost-effective option is being used, thus ensuring rational use of public resources. |
URL | https://www.nice.org.uk/guidance/cg70/resources/surveillance-report-2017-induction-of-labour-2008-ni... |
Description | INSERM workshop 234: Network meta-analysis |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Policymakers/politicians |
Results and Impact | 2-day workshop for French health policy makers and technical staff, which generated interest in the methodology developed and disseminated findings. It is likely that the methods being developed will be used in France. |
Year(s) Of Engagement Activity | 2015 |
URL | http://atelier234.clinicalepidemio.fr/ |
Description | RSS Conference |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Invited speaker at annual conference of the Royal Statistical Society. Session was very well attended with standing room only and discussion held afterwards with some attendees may lead to future collaborations. This has also lead me to be invited to be a plenary speaker at the 2016 Conference of the International Social for Clinical Biostatisticians, to be held in August. |
Year(s) Of Engagement Activity | 2015 |
URL | http://www.rss.org.uk/RSS/Events/RSS_Conference/2015_Conference/RSS/Events/Conference/2015_conferenc... |