Obtaining valid and reliable estimates of treatment effectiveness from evidence synthesis of psychiatric trials.

Lead Research Organisation: University of Bristol
Department Name: Community-Based Medicine

Abstract

Establishing the most effective treatment(s) for depression is a priority for population health. By 2020 depression will be the second highest cause of disease worldwide with the cost to the NHS currently around #9 billion. Health Technology Assessment (HTA) supports healthcare decision-making through the evaluation of the clinical and cost-effectiveness of medical treatments and the publication of treatment recommendations and guidelines for the NHS. Gold standard evidence for HTA comes from meta-analysis of randomised controlled trials (RCTs). A meta-analysis combines the data from multiple RCTs for the same condition in one analysis to produce more powerful evidence of whether a treatment is effective or not. However, empirical research suggests that RCTs in psychiatry are particularly susceptible to bias, which can lead to under- or over-estimation of treatment effectiveness. New methods for controlling for bias in meta-analysis have been proposed which generate un-biased estimates of treatment effectiveness. This project evaluates the potential benefits and reliability of these new methods compared to standard meta-analysis. The project uses these methods to evaluate the effectiveness of new-generation antidepressants, psychotherapies and older treatments, such as tricyclic antidepressants, to answer the question which is the most effective treatment for depression?

Technical Summary

Background: Establishing the most effective treatment(s) for depression is a population health priority. By 2020 depression will be the second highest cause of disease worldwide, with the cost to the NHS around #9 billion. Evidence synthesis using randomised controlled trials (RCTs) underpins Health Technology Assessment (HTA) and clinical guidelines. However, empirical evidence suggests psychiatric RCTs are susceptible to bias, raising questions about reliability and validity of effect estimates from psychiatric HTA. Two recent methodological developments make it possible to control for bias in psychiatric HTA; Bias-adjusted meta-analysis and Mixed Treatment Comparisons.

Aims & Objectives: to examine the suitability of MTC for psychiatric RCTs and assess how biases in evidence synthesis of psychiatric RCTs can be controlled and mitigated to generate more precise, valid estimates of treatment effect. I will;
1. explore the benefits of bias-adjusted MTC compared to standard meta-analysis in the presence of quality-related biases and effect modifiers,
2. assess the effect of extending an MTC network of evidence for depression to include all relevant treatments and assess the ‘risk of bias‘ for each included RCT;
3. explore and compare the effects of bias adjustment on treatment effects estimates in the extended depression network of evidence, using approaches proposed by Turner, Welton and Dias.

Design & Methodology: A review of NICE guidelines and Cochrane reviews will be conducted to identify and assemble psychiatric MTC networks of evidence. The networks inform a simulation study to examine the reliability of bias-adjustment, comparing effect estimates from meta-analysis, unadjusted MTC and bias-adjusted MTC in the presence of bias. The substantive part of the project extends an existing network of new-generation antidepressants for depression, including additional treatments and assessing each RCT for bias using the Cochrane risk of bias tool. Theended network forms the basis for an empirical study to explore and compare the effects of bias-adjusted MTC on depression effect estimates, using three recently proposed bias-adjustment methods. Analyses use a Bayesian approach implemented in WinBUGS.

Scientific Opportunities: Project outputs will
generate validated, reliable treatment effect estimates and rankings for depression;
inform a future grant proposal of bias-adjusted MTC for all depression treatments to identify the most effective treatment overall;
evaluate the benefits of bias-adjusted MTC compared to meta-analysis in psychiatric HTA
inform NICE‘s research agenda regarding the validity and reliability of MTC;
identify whether the three MTC bias-adjustment methods can be combined into a single approach.

Publications

10 25 50
publication icon
Ades AE (2013) Evidence synthesis for decision making 7: a reviewer's checklist. in Medical decision making : an international journal of the Society for Medical Decision Making

publication icon
Alfirevic Z (2016) Methods to induce labour: a systematic review, network meta-analysis and cost-effectiveness analysis. in BJOG : an international journal of obstetrics and gynaecology

publication icon
Caldwell DM (2015) Extending Treatment Networks in Health Technology Assessment: How Far Should We Go? in Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research

 
Description Use of material within a postgraduate course
Geographic Reach Local/Municipal/Regional 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Health Technology Assessment
Amount £261,107 (GBP)
Organisation National Institute for Health Research 
Sector Public
Country United Kingdom
Start 09/2016 
End 12/2018
 
Description NIHR Public Health Research
Amount £296,668 (GBP)
Organisation National Institute for Health Research 
Sector Public
Country United Kingdom
Start 10/2016 
End 11/2018
 
Description CIHR Drug Safety and Effectiveness network Knowledge Synthesis 
Organisation Ottawa Hospital Research Institute
Country Canada 
Sector Academic/University 
PI Contribution Consultant for network meta-analysis projects/ steering board
Collaborator Contribution Submitted the research grant, and provide methodological problems similar to the fellowship project i am working on
Impact none yet
Start Year 2014
 
Description Cochrane Collaboration Comparing Multiple Interventions Methods Group 
Organisation Tufts Medical Center
Country United States 
Sector Academic/University 
PI Contribution As a result of the funding i receive via my fellowship I was able to form links within the Cochrane Collaboration to form a new methods group examining how to evaluate multiple interventions within systematic reviews. I am a co-convenor of this methods group.
Collaborator Contribution Collaboration within this group facilitates my goal to disseminate the methodology to a wider audienceCollaboration within this group facilitates my goal to disseminate the methodology to a wider audience and to explore the impact of bias within network meta-analysisCollaboration within this group facilitates my goal to disseminate the methodology to a wider audience.
Impact The collaboration is multi-disicplinary, involving statisticians, methodologists and clinicians. No outputs to report this year. I anticipate outcomes in the later stages of the fellowship period.
Start Year 2010
 
Description Cochrane Collaboration Comparing Multiple Interventions Methods Group 
Organisation University of Cambridge
Department MRC Biostatistics Unit
Country United Kingdom 
Sector Public 
PI Contribution As a result of the funding i receive via my fellowship I was able to form links within the Cochrane Collaboration to form a new methods group examining how to evaluate multiple interventions within systematic reviews. I am a co-convenor of this methods group.
Collaborator Contribution Collaboration within this group facilitates my goal to disseminate the methodology to a wider audienceCollaboration within this group facilitates my goal to disseminate the methodology to a wider audience and to explore the impact of bias within network meta-analysisCollaboration within this group facilitates my goal to disseminate the methodology to a wider audience.
Impact The collaboration is multi-disicplinary, involving statisticians, methodologists and clinicians. No outputs to report this year. I anticipate outcomes in the later stages of the fellowship period.
Start Year 2010
 
Description Cochrane Collaboration Comparing Multiple Interventions Methods Group 
Organisation University of Ioannina
Department Department of Hygiene and Epidemiology
Country Greece 
Sector Academic/University 
PI Contribution As a result of the funding i receive via my fellowship I was able to form links within the Cochrane Collaboration to form a new methods group examining how to evaluate multiple interventions within systematic reviews. I am a co-convenor of this methods group.
Collaborator Contribution Collaboration within this group facilitates my goal to disseminate the methodology to a wider audienceCollaboration within this group facilitates my goal to disseminate the methodology to a wider audience and to explore the impact of bias within network meta-analysisCollaboration within this group facilitates my goal to disseminate the methodology to a wider audience.
Impact The collaboration is multi-disicplinary, involving statisticians, methodologists and clinicians. No outputs to report this year. I anticipate outcomes in the later stages of the fellowship period.
Start Year 2010
 
Description Cochrane Depression, Anxiety and Neurosis group 
Organisation The Cochrane Collaboration
Department Depression Anxiety and Neurosis Group (CCDAN)
Country United Kingdom 
Sector Charity/Non Profit 
PI Contribution I have collaborated on systematic reviews, which in turn will inform my fellowship award
Collaborator Contribution The data gathered from the systematic reviews will inform my fellowship project
Impact 1. HIRED MAP systematic review team: including Caldwell, DM. Mindfulness-based 'third wave' cognitive and behavioural therapies versus other psychological therapies for depression. Cochrane Database of Systematic Reviews 2013 2. HIRED MAP systematic review team: including Caldwell, DM. Mindfulness-based 'third wave' cognitive and behavioural therapies versus treatment as usual for depression. Cochrane Database of Systematic Reviews 2013 3. Shinohara K, Honyashiki M, Imai H, Churchill R, Hunot V, Caldwell DM, Davies P, Moore T, Furukawa TA. Behavioural therapies versus other psychological therapies for depression. Cochrane Database of Systematic Reviews 2013. In press.
Start Year 2008
 
Description Approaches for synthesis of complex interventions: components-based network meta-analysis 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact Invited presentation given on approaches for synthesis of complex interventions: components-based network meta-analysis. An invited presentation on the 2016 paper published in evidence based mental health - Caldwell DM & Welton NJ. Approaches for synthesizing complex mental-health interventions in meta-analysis. Evidence Based Mental Health. 2016; 19:16-21. Audience invited from across London universities.
Year(s) Of Engagement Activity 2017
URL http://eppi.ioe.ac.uk/cms/Default.aspx?tabid=3471
 
Description Caldwell, DM. Network meta-analysis: value and importance to Cochrane. Cochrane Common Mental Disorders group - 20th Anniversary meeting, York, July 2016 (Invited) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Caldwell, DM. "Network meta-analysis: value and importance to Cochrane". Cochrane Common Mental Disorders group - 20th Anniversary meeting, York, July 2016 (Invited presentation)
Year(s) Of Engagement Activity 2016
 
Description World Health Organisation (WHO): strengthening the process and methods for retrieval, synthesis and assessment of evidence on complex, multidisciplinary interventions for WHO guidelines 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Third sector organisations
Results and Impact Review of 6 background papers commissioned to strengthen WHO guideline development process to improve synthesis, assessment and application of evidence on complex and multidisciplinary interventions. There are 5 themes for the meeting. Deborah Caldwel is co-author of draft for quantitative synthesis theme (lead Julian Higgins).
Year(s) Of Engagement Activity 2017
URL http://www.who.int/kms/scopingmeetingrsaecmi/en/