Multi-parameter Evidence Synthesis

Lead Research Organisation: MRC Health Services Resrch Collaboration

Abstract

It is essential that treatment recommendations and public health policy decisions made by the Department of Health and the NHS are able to attract widespread support. This requires that the methodology used to reach decisions is transparent, open to debate, and able to attract consensus among manufacturers, managers, doctors, and experts.||These decisions rely on models. Model credibility, in turn, requires that all the available evidence is used to drive the model, and that all uncertainties in the evidence are propagated carefully through the model.||The role of this programme is to advance the methodology for evidence synthesis, so that it matches these aspirations. Critical issues are: how to combine multiple sources of evidence informing multiple functions of model parameters, and how to detect and address inconsistencies in the evidence base. If multiple, interlocking evidence sources can be shown to be consistent, then the ability to incorporate information on functions of parameters will increase the precision of model parameters, reduce uncertainty in decision, enhance the credibility of decisions, and reduce the need for further research.|||

Technical Summary

Our focus is the development and dissemination of statistical methods for evidence synthesis. Standard meta-analysis has been seen as a way of summarising the literature on a single parameter. However, decision problems may involve several treatments, and the evidence available may involve a range of different outcomes, in studies of different designs. In particular, evidence may be available both on model parameters and on multiple functions of parameters. Therefore, in contrast to standard decision analytic methods, in which each parameter is informed by a separate study or meta-analysis, there are invariably either fewer, or (more often) more data sources than there are parameters, leading to a redundancy of information. This creates an opportunity to assess whether different sources of data are consistent with each other, and thus to validate decision models in a formal way. ||The programme has five planks:||1. Synthesis issues raised by generic evidence structures. We are working on: Mixed treatment comparisons; Markov models, and multiple and surrogate outcomes; treatment combinations; and epidemiological models for prevalence and incidence. In choosing which structures to work on, we give emphasis to synthesis problems recurring at the NICE appraisals process. In addition, collaborations in epidemiology and public health (below) expose us to further evidence structures. ||2. Recurring theoretical problems. Evidence consistency is a major focus. We are investigating a range of global and local diagnostics to identify inconsistency between evidence sources, given a particular model. A second focus is Expected Value of Information methodology. The challenge is to devise computational methods that are compatible with the correlated parameter distributions output from MPES. ||3. Substantive projects. Alongside the MRC programme are a series of substantive projects, funded independently at Bristol, or involving collaborations outside: incidence and prevalence of HIV in the UK, the prevalence of HCV and drug use, toxoplasmosis incidence, screening for congenital toxoplasmosis, screening for Group B Streptococcus. ||4. Dissemination strategy. In publications and conference attendance, we separately target 3 communities: health economics/decision analysis, epidemiology/biostatistics, and EBM/systematic review, in communities . Our five-day course Evidence Synthesis for Decision Making, jointly run with colleagues at Leicester, which will have had three runs during 2005, allows both groups to show-case practical worked examples to CEA practitioners, statisticians, and systematic reviewers. ||5. A national collaborative network Beyond the bilateral collaborative relationships, we are attempting to grow a wider collaboration - eventually with a broader funding base - as an efficient way to develop and refine these methods and increase their use, based on a wider strategy for development of MRC funding Health Services Research in the UK.

Publications

10 25 50

publication icon
Ades AE (2008) Multiparameter evidence synthesis in epidemiology and medical decision-making. in Journal of health services research & policy

publication icon
Ades AE (2010) Network meta-analysis with competing risk outcomes. in Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research

publication icon
Ades AE (2004) Expected value of sample information calculations in medical decision modeling. in Medical decision making : an international journal of the Society for Medical Decision Making

publication icon
Ades AE (2005) The interpretation of random-effects meta-analysis in decision models. in Medical decision making : an international journal of the Society for Medical Decision Making

publication icon
Ades AE (2011) Indirect and mixed treatment comparisons in arthritis research. in Rheumatology (Oxford, England)

publication icon
Ades AE (2013) Evidence synthesis for decision making 7: a reviewer's checklist. in Medical decision making : an international journal of the Society for Medical Decision Making

 
Description Short Courses
Geographic Reach Multiple continents/international 
Policy Influence Type Influenced training of practitioners or researchers
Impact The programme has lead to the development of the 5-day course "Bayesian Evidence Synthesis for Decision Making" and the 3-day course "Indirect and Mixed Treatment Comparisons", which have been run once each year since 2004, with our collaborators at University of Leicester (Prof Keith Abrams, Prof Alex Sutton, Dr Nicola Cooper). These course have been given to 35-48 people on each occasion, an equal mixture of academic, government, and industry statisticians and health economists. We have also run additional 2- or 3-day courses for Pfizer and for National Institute of Health & Clinical Excellence (NICE) Clinical Guidelines Group. In addition we have run 1-day introductory courses on Mixed Treatment Comparisons for NICE, and Janssen-Cilag.
 
Description MRC G0801947
Amount £150,000 (GBP)
Funding ID G0801947 
Organisation Medical Research Council (MRC) 
Sector Academic/University
Country United Kingdom
Start 07/2009 
End 07/2011
 
Description MRC G0901488
Amount £200,000 (GBP)
Funding ID G0901488 
Organisation Medical Research Council (MRC) 
Sector Academic/University
Country United Kingdom
Start 12/2009 
End 11/2011
 
Description MRC Hubs for Trial Methodology Research (HTMR)
Amount £250,000 (GBP)
Organisation Medical Research Council (MRC) 
Sector Academic/University
Country United Kingdom
Start 01/2009 
End 01/2014
 
Description MRC Methodology Research Panel
Amount £250,000 (GBP)
Organisation Medical Research Council (MRC) 
Sector Academic/University
Country United Kingdom
Start 08/2007 
End 07/2010
 
Description MRC NIQUAD Cluster, Addiction
Amount £200,000 (GBP)
Organisation Medical Research Council (MRC) 
Sector Academic/University
Country United Kingdom
Start 03/2010 
End 02/2013
 
Description MRC Population Health Sciences Research Network (PHSRN) funding
Amount £90,000 (GBP)
Organisation Medical Research Council (MRC) 
Sector Academic/University
Country United Kingdom
Start 09/2007 
End 08/2010
 
Description NICE contract to supply Technical Support Unit
Amount £500,000 (GBP)
Organisation National Institute for Health and Care Excellence (NICE) 
Sector Public
Country United Kingdom
Start 05/2011 
End 04/2014
 
Description NIHR HTA (08/60)
Amount £10,000 (GBP)
Funding ID HTA/08/60/01 
Organisation National Institute for Health Research 
Department Health Technology Assessment Programme (HTA)
Sector Public
Country United Kingdom
Start 09/2009 
End 08/2010
 
Description NIHR HTA (08/70)
Amount £20,000 (GBP)
Funding ID HTA/08/70/01 
Organisation National Institute for Health Research 
Department Health Technology Assessment Programme (HTA)
Sector Public
Country United Kingdom
Start 07/2009 
End 06/2010
 
Description NIHR Public Health (Cancer Screening)
Amount £40,000 (GBP)
Organisation National Institute for Health Research 
Sector Public
Country United Kingdom
Start 01/2009 
End 12/2013
 
Description NIHR Public Health (Domestic Violence)
Amount £100,000 (GBP)
Organisation National Institute for Health Research 
Sector Public
Country United Kingdom
Start 01/2009 
End 12/2013
 
Title Estimates of HIV prevalence in the UK 
Description This is a methodology for estimating the numbers of HIV infected individual - including those not diagnosed - based on a wide range of inter-locking sources of data, both routine and ad hoc surveys. Unlike the "direct method" used previously The methods allow for appropriate statistical inference, uncertainty propagation and consistency checking (validation). The original work was carried out under an MRC Special Project grant funded under CESA, within the Bristol MPES programme. The software is openly available 
Type Of Material Data analysis technique 
Year Produced 2007 
Provided To Others? Yes  
Impact These methods have been used by the Health Protection Agency every year since 2007 to provide the official Department of Health estimates 
 
Title WinBUGs software for Evidence Synthesis 
Description The software described here grew out of the Multi-Parameter Evidence Synthesis programme, initially funded by MRC HSRC. The software developed over several years, and was developed by Nicky Welton, Guobing Lu, and Sofia Dias, who were all funded by MRC. This software is available on the NICE Decision Support Unit Website (http://www.nicedsu.org.uk/) in the form of 7 Technical Support Documents on Evidence Synthesis. These TSDs describe a core model that carries out pair-wise meta-analysis, synthesis of multi-arm trials, network meta-analysis and indirect comparisons without distinction (TSD-2) The core model can be applied, in a modular way, to a wide range of Generalized Linear Models (TSD-2) Software is also provided that can be used to check inconsistency in networks (TSD-4), and meta-regression in networks (TSD-3). The same core software is available to be downloaded from the ISPOR website (International Society for Pharmaco-economic and Outcomes Research). It can be found in teh "Appendix" to http://www.sciencedirect.com/science/article/pii/S1098301511013283 
Type Of Material Improvements to research infrastructure 
Year Produced 2011 
Provided To Others? Yes  
Impact The software offers users a methodology that conforms to the requirements set out in the 2008, 2012 NICE Methods Guides. It also conforms to the specifications described in the ISPOR Task Force report on Indirect comparisons and Network Meta-analysis This software is used in a very large proportion of Submissions to the National Institute of Health and Clinical Excellence Technical Appraisals process: specifically those that involve network meta-analysis. It is also used widely in NICE Clinical Guideline development 
URL http://www.nicedsu.org.uk/
 
Description Leicester 
Organisation University of Leicester
Department Department of Epidemiology and Public Health
Country United Kingdom 
Sector Academic/University 
PI Contribution see collabator details
Collaborator Contribution Collaborators (Prof Keith Abrams, Prof Alex Sutton, Dr Nicola Cooper have contributed intellectually to a number of our lines of research. This is reflected in a number of co-authored publications. The collaborators expertise in evidence synthesis and Bayesian methods made them excellent partners in the short course programme we now jointly run. One of the MRC HSRC studentships was placed in Univ Leicester, with Prof Alex Sutton as the main supervisor, I was a co-supervisor. Publications are listed in first section.
Impact listed in main section
 
Description MRC Biostatistics 
Organisation University of Cambridge
Department MRC Biostatistics Unit
Country United Kingdom 
Sector Public 
PI Contribution We have contributed theory and illustrative analyses on use of network meta-analysis in identifying and adjusting for types of bias in RCTs We have contributed methodology on assessment of inconsistency in networks of RCTs
Collaborator Contribution Collaboration with Julian Higgins has lead to our obtaining MRC funding (Population Health Sceince Research Network 14) for "optimisIng INFORMATION FROM rANDOMISED cONTROLLED tRIALS It has also lead to collaborative work on Inconsistency with JUlian Higgins and Ian White It has also lead to work with Julian Higgins, Ian White and others on Missing data in meta-analysis
Impact see publications list, publications with S Dias as first author
 
Description NICE - ABPI Masterclassess 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? Yes
Type Of Presentation Keynote/Invited Speaker
Geographic Reach National
Primary Audience Health professionals
Results and Impact We have put on the Evidence Synthesis part of general masterclasses on "what make a good submission", and we have also designed and presented masterclasses that focus entirely on evidence synthesis in the NICE context

.
Year(s) Of Engagement Activity 2009,2010,2011,2012
 
Description NICE Clincial Guidelines 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? Yes
Type Of Presentation Keynote/Invited Speaker
Geographic Reach National
Primary Audience Health professionals
Results and Impact As a result it was possible for guideline development groups to make coherent clinical and cost-effectiveness comparisons between several treatments, based on all the available evidence.
This is not possible with the use of methods we have pioneered

Partly arising from this work, the group has won a 3-year contract to supply a Technical Support Unit to the NICE Clinical Guidelines development process
Year(s) Of Engagement Activity 2009,2010,2011