Expected Value of Information and Synthesis Methods for Research Prioritisation and Study Design

Lead Research Organisation: University of Bristol
Department Name: Community-Based Medicine

Abstract

At present there is a gap between the way decisions are made about whether to adopt new treatments for use in the NHS, and the way decisions are made about whether more research is needed on these treatments. As a result, resources may be wasted researching treatments that were never likely to be cost-effective, or by adopting treatments which, if more evidence were collected, would not be cost-effective. It is possible to calculate the probability that a decision to adopt a treatment on the basis of current evidence would turn out to be the wrong decision if more evidence was collected, and how much resource waste and loss of quality of life this is likely to represent. These calculations provide an objective and fair method for informing the decision of which studies to fund, and how best to design those studies. The methods are, however, complex. This project aims to find ways of making the calculations easier so that they can be used more widely to help research funding bodies commission research that will help make better treatment decisions for patients in the NHS.

Technical Summary

The National Institute for Health and Clinical Excellence (NICE) and similar bodies worldwide use formal modelling techniques to answer policy questions about whether to adopt a new technology for routine use. But a series of further policy issues then arise that are relevant to research commissioning bodies such as the National Institute for Health Research (NIHR) Health Technology Assessment Programme (NCCHTA) and the MRC: (a) is more evidence required to support the adoption of this technology (b) which model inputs in particular should we collect evidence on? (c) what research designs deliver the best societal pay-off from research? (d) what is the optimal sequence of multiple new studies? These questions can be addressed using Expected Value of Information (EVI) methods.

While the principles of EVI methods are well understood, applications in health and medicine face technical challenges before they can be routinely implemented. These include: correlated model inputs; non-linear decision models; complex relationships between evidence and model inputs; heterogeneity in baseline characteristics, subgroups, and model uncertainty; biomarkers and biased evidence; optimisation over large design-spaces and optimal sequences of research. Changes in the policy environment, leading to coverage with evidence and risk-sharing schemes raise further technical challenges.

The aim of this proposal is to develop methods that address these technical challenges for EVI computation that are practical within the time constraints faced by NICE, NIHR NCCHTA, and those designing trials, and sensitive to the developing policy context. To achieve this, a series of carefully chosen worked illustrations will be assembled, covering the most important and commonly encountered evidence structures and cost-effectiveness models. The examples will include the design of randomised controlled trials, epidemiological studies, drug-development studies, use of biomarkers, and design of research portfolios.

EVI analysis depends on the validity of underlying systematic review, evidence synthesis, and cost-effectiveness analysis, and has implications for all of these disciplines. The training element is designed to broaden my knowledge of these disciplines, and will expose me to policy considerations at National level surrounding the use of these methods in Health Technology Assessment. Thus ensuring the policy-relevance of the research, and facilitating future research across the whole EVI agenda.

Research findings will be disseminated through publications and presentations at international conferences. Wherever possible generic methods to prioritise and design new trials/studies will be made available for use in standard software. The worked exemplars and methods will form the basis of short courses and a book.
400 words

Publications

10 25 50
publication icon
Ades AE (2011) Indirect and mixed treatment comparisons in arthritis research. in Rheumatology (Oxford, England)

publication icon
Ades AE (2013) Evidence synthesis for decision making 7: a reviewer's checklist. in Medical decision making : an international journal of the Society for Medical Decision Making

publication icon
Ades AE (2010) Network meta-analysis with competing risk outcomes. in Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research

publication icon
Alfirevic Z (2016) Methods to induce labour: a systematic review, network meta-analysis and cost-effectiveness analysis. in BJOG : an international journal of obstetrics and gynaecology

publication icon
Alfirevic Z (2015) Labour Induction With Prostaglandins A Systematic Review and Network Meta-Analysis in Obstetrical & Gynecological Survey

publication icon
Astin MP (2015) Diagnostic value of symptoms of oesophagogastric cancers in primary care: a systematic review and meta-analysis. in The British journal of general practice : the journal of the Royal College of General Practitioners

 
Description Advanced Breast Caner Guideline
Geographic Reach National 
Policy Influence Type Membership of a guidance committee
 
Description InterTASC workshop presentation
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Masterclass for NICE Decision Support Unit/ABPI on What makes a good submission to NICE?
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
Impact The training helps pharmaceutical companies prepare good submissions to NICE to enable NICE to better appraise their health technologies. This means that clearer reimbursement decisions can be made, benefiting patients and healthcare providers.
 
Description NICE Evidence Synthesis TSDs
Geographic Reach Multiple continents/international 
Policy Influence Type Citation in other policy documents
Impact Guidance on methodology for decisions made by NICE Technology Appraisals Committees. These Technical Support Documents are recommended in the NICE Methods Guide, and are cited widely in submissions to NICE.
URL http://www.nicedsu.org.uk/evidence-synthesis-tsd-series(2391675).htm
 
Description NICE Techology Appraisal committee member
Geographic Reach National 
Policy Influence Type Membership of a guidance committee
Impact The guidance issued from the NICE Technology Appraisals committee are mandatory within 3 months, and therefore have direct impact. The guidance is based on an assessment of cost-effectiveness, and so improves efficiency of health care
 
Description Schizophrenia Clinical Guideline
Geographic Reach National 
Policy Influence Type Membership of a guidance committee
 
Description Short Course on Bias Adjustment in Meta-Analysis and Network Meta-Analysis
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
Impact Training course tailored to the needs of technical teams on NICE clinical guidelines.
 
Description Short Course on Statistical Methods for Value of Information Analysis
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Short course Inconsistency in network meta-analysis
Geographic Reach Local/Municipal/Regional 
Policy Influence Type Influenced training of practitioners or researchers
Impact Training was given in-house to ICON plc who are consultants tot he pharmaceutical industry, helping them prepare submissions to NICE.
 
Description Short course on Meta-Analysis of Diagnostic Test Accuracy Studies
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Short courses in Mixed Treatment Comparisons
Geographic Reach Multiple continents/international 
Policy Influence Type Influenced training of practitioners or researchers
Impact Run many courses on indirect and mixed treatment comparison methods in the UK, abroad, and for industry. The methods are now widely used in submissions to NICE, and in the development of NICE Clinical Guidelines
 
Description Urinary Incontinence Clinical Guideline
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
 
Description 08/60/01 Electronic aides to smoking cessation research grant
Amount £91,015 (GBP)
Organisation National Institute for Health Research 
Sector Public
Country United Kingdom
Start 04/2009 
End 03/2010
 
Description 08/70/01 value of information for IVIG research grant
Amount £126,799 (GBP)
Organisation National Institute for Health Research 
Sector Public
Country United Kingdom
Start 05/2009 
End 04/2010
 
Description 10/98 Whole body physiotherapy package for hypermobility call
Amount £329,881 (GBP)
Organisation National Institute for Health Research 
Sector Public
Country United Kingdom
Start 09/2012 
End 06/2015
 
Description Development of a method for adjusting trial results for biases in meta-analysis: combining generic evidence on bias with detailed trial assessment.
Amount £274,851 (GBP)
Funding ID MC_EX_MR/K014587/1 
Organisation Medical Research Council (MRC) 
Sector Academic/University
Country United Kingdom
Start 06/2013 
End 06/2016
 
Description Effectiveness and cost-effectiveness of new anticoagulants for venous thromboembolism and sroke prevention in atrial fibrillation.
Amount £300,355 (GBP)
Organisation National Institute for Health Research 
Department Health Technology Assessment Programme (HTA)
Sector Public
Country United Kingdom
Start 10/2013 
End 12/2014
 
Description MICA: Model Based Network Meta-Analysis for Pharmacometrics and Drug-Development.
Amount £201,060 (GBP)
Funding ID MR/M005615/1 
Organisation Medical Research Council (MRC) 
Sector Academic/University
Country United Kingdom
Start 11/2014 
End 10/2017
 
Description MRC ConDuCT-II Hub for Trial Methodology Research
Amount £1,990,000 (GBP)
Organisation Medical Research Council (MRC) 
Sector Academic/University
Country United Kingdom
Start 04/2014 
End 03/2019
 
Description MRC Fellowship: Sarah Donegan
Amount £26,618 (GBP)
Organisation Medical Research Council (MRC) 
Sector Academic/University
Country United Kingdom
Start 07/2013 
End 06/2017
 
Description MRC Hub for Trials Methodology Research Network (biologics)
Amount £7,532 (GBP)
Organisation Medical Research Council (MRC) 
Sector Academic/University
Country United Kingdom
Start 09/2010 
End 09/2010
 
Description MRC Hubs for Trial Methodology Network
Amount £8,566 (GBP)
Organisation Medical Research Council (MRC) 
Sector Academic/University
Country United Kingdom
Start 11/2011 
End 03/2012
 
Description MRC research grant Chlamydia G0801947
Amount £173,500 (GBP)
Funding ID G0801947 
Organisation Medical Research Council (MRC) 
Sector Academic/University
Country United Kingdom
Start 06/2009 
End 05/2011
 
Description MRC research grant G1001338 synthesis of time to event data
Amount £145,106 (GBP)
Funding ID G1001338 
Organisation Medical Research Council (MRC) 
Sector Academic/University
Country United Kingdom
Start 12/2010 
End 06/2013
 
Description Meta-Modelling for Value of Information Calculations Workshop.
Amount £9,160 (GBP)
Organisation Medical Research Council (MRC) 
Department Network of Hubs for Trials Methodology Research (HTMR)
Sector Academic/University
Country United Kingdom
Start 04/2014 
End 09/2014
 
Description NICE Clinical Guidelines Technical Support Unit
Amount £313,872 (GBP)
Organisation National Institute for Health and Care Excellence (NICE) 
Sector Public
Country United Kingdom
Start 03/2010 
End 02/2013
 
Description NICE Clinical Guidelines Technical Support Unit
Amount £327,489 (GBP)
Organisation National Institute for Health and Care Excellence (NICE) 
Sector Public
Country United Kingdom
Start 04/2016 
End 03/2019
 
Description NIHR Health Protection Unit: Evaluation of Interventions
Amount £3,865,761 (GBP)
Funding ID HPRU-2012-10026 
Organisation National Institute for Health Research 
Sector Public
Country United Kingdom
Start 04/2014 
End 03/2019
 
Description NIHR Programme Grant for Applied Research
Amount £2,233,801 (GBP)
Funding ID RP-PG-0514-20012 
Organisation National Institute for Health Research 
Sector Public
Country United Kingdom
Start 04/2016 
End 03/2022
 
Description Research grant mapping outcomes G0901488
Amount £232,052 (GBP)
Funding ID G0901488 
Organisation Medical Research Council (MRC) 
Sector Academic/University
Country United Kingdom
Start 01/2010 
End 12/2012
 
Description Screening Strategies for Atrial Fibrillation: A Systematic Review and Cost-Effectiveness Analysis.
Amount £181,379 (GBP)
Organisation National Institute for Health Research 
Sector Public
Country United Kingdom
Start 10/2014 
End 03/2016
 
Description The choice between hip prosthetic bearing surfaces in total hip replacement: evidence synthesis, statistical analysis and decision modelling to evaluate the effectiveness and cost-effectiveness of prosthetic implants for the NHS.
Amount £230,304 (GBP)
Funding ID PB-PG-0613-31032 
Organisation National Institute for Health Research 
Department Research for Patient Benefit
Sector Public
Country United Kingdom
Start 07/2014 
End 06/2016
 
Description The effectiveness, acceptability and cost-effectiveness of psychosocial interventions for maltreated children and adolescents: An evidence synthesis.
Amount £335,124 (GBP)
Organisation National Institute for Health Research 
Department Health Technology Assessment Programme (HTA)
Sector Public
Country United Kingdom
Start 02/2013 
End 07/2014
 
Description What are the indications for prescribing antidepressants that will lead to a clinical benefit? NIHR programme grant.
Amount £1,999,968 (GBP)
Organisation National Institute for Health Research 
Sector Public
Country United Kingdom
Start 03/2012 
End 02/2017
 
Description What interventions improve outcomes for children exposed to domestic abuse: a systematic review, evidence synthesis, and research recommendations.
Amount £242,390 (GBP)
Organisation National Institute for Health Research 
Department Health Technology Assessment Programme (HTA)
Sector Public
Country United Kingdom
Start 03/2013 
End 08/2014
 
Description Which method is best for the induction of labour: A systematic review, network meta-analysis and cost effectiveness analysis.
Amount £162,371 (GBP)
Organisation National Institute for Health Research 
Department Health Technology Assessment Programme (HTA)
Sector Public
Country United Kingdom
Start 09/2013 
End 12/2014
 
Description programme grant RP-PG-0609-10171 Negative pressure wound dressings
Amount £13,820 (GBP)
Organisation National Institute for Health Research 
Sector Public
Country United Kingdom
Start 06/2011 
End 05/2015
 
Description Ashma Krishan PhD 
Organisation University of Liverpool
Country United Kingdom 
Sector Academic/University 
PI Contribution Supervision
Collaborator Contribution Supervision
Impact Non yet
Start Year 2015
 
Description Becky Boucher fellowship 
Organisation University of Leicester
Country United Kingdom 
Sector Academic/University 
PI Contribution Provide advice on statistical methodology
Collaborator Contribution Fellowship programme of work, publications.
Impact None yet - just started
Start Year 2014
 
Description Covariates in NMA 
Organisation University of Liverpool
Country United Kingdom 
Sector Academic/University 
PI Contribution Provide expertise on network meta-analysis
Collaborator Contribution fellowship, publications, datasets
Impact Publications in preparation.
Start Year 2013
 
Description Laura Flight Fellowship 
Organisation University of Sheffield
Country United Kingdom 
Sector Academic/University 
PI Contribution Advisory
Collaborator Contribution Supervision
Impact Non yet
Start Year 2015
 
Description Model Based Network Meta-Analysis 
Organisation Pfizer Ltd
Country United Kingdom 
Sector Private 
PI Contribution Successful MRC grant, that is just about to begin. Pfizer bring model based meta-analysis expertise, and Bristol bring network meta-analysis skills. Funds come jointly from MRC MICA Methodology Research grant and from pfizer.
Collaborator Contribution Pfizer bring model based meta-analysis expertise and and access to datasets and relevant examples.
Impact MRC MICA grant
Start Year 2014
 
Description Tracey Sachs fellowship 
Organisation University of East Anglia
Country United Kingdom 
Sector Academic/University 
PI Contribution Provide advice on expected value of information computation
Collaborator Contribution Tracey Sach fellowship: papers, research
Impact Only just started - no outputs yet.
Start Year 2014
 
Description Value of Information Methods Group 
Organisation University College London
Country United Kingdom 
Sector Academic/University 
PI Contribution Arranged and funded meetings. Developed materials for workshop. Organised and developed materials for short course.
Collaborator Contribution Developed materials for workshop. Organised and developed materials for short course.
Impact Welton NJ, Baio G, Strong M, Heath A. The role of value of information in trial design: are we missing an opportunity? Workshop. 3rd International Clinical Trials Methodology Conference, Glasgow 16th-17th November 2015 Short course on Statistical Methods for the Value of Information Analysis. 8-10 June 2016, University College London
Start Year 2014
 
Description Value of Information Methods Group 
Organisation University of Sheffield
Country United Kingdom 
Sector Academic/University 
PI Contribution Arranged and funded meetings. Developed materials for workshop. Organised and developed materials for short course.
Collaborator Contribution Developed materials for workshop. Organised and developed materials for short course.
Impact Welton NJ, Baio G, Strong M, Heath A. The role of value of information in trial design: are we missing an opportunity? Workshop. 3rd International Clinical Trials Methodology Conference, Glasgow 16th-17th November 2015 Short course on Statistical Methods for the Value of Information Analysis. 8-10 June 2016, University College London
Start Year 2014
 
Description Evidence Synthesis MasterClass 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact Gave session and practical "What makes a good submission to NICE? Evidence synthesis." NICE/Decision Support Unit/ ABPI (Association of the British Pharmaceutical Industry) Masterclass.
Year(s) Of Engagement Activity 2015,2016
 
Description ISPOR Workshop The role of value of information in HTA: are we missing an opportunity? 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Workshop delivered at ISPOR conference attended by policy makers, industry and academics. The workshop led to a lively debate, and requests for information.
Year(s) Of Engagement Activity 2015
 
Description Indirect & Mixed Treatment Comparisons. All Wales Medicines Strategy Group Training 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Policymakers/politicians
Results and Impact Gave invited presentation on network meta-analysis to the All Wales Medicines Strategy Group Training Day to provide training in their methods of technology appraisals
Year(s) Of Engagement Activity 2015
 
Description Presentation at NICE Away Day 2013 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? Yes
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Made NICE Committee members and Evidence Review Groups aware of the potential for the use of external information to better inform the shape of survival curves, to help make better reimbursement decisions.

The survival curve reconstruction methods have been used in submissions to NICE technology appraisals. The committee members have an improved understanding of the key issues in extrapolating beyond trial follow-up periods.
Year(s) Of Engagement Activity 2013
 
Description Value of information and its potential role in guidelines. NICE-NCC Technical Meeting 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact Gave invited presentation on the role of value of information in guidelines to NICE National Coordinating Centre Technical staff.
Year(s) Of Engagement Activity 2014