MICA: Model Based Network Meta-Analysis for Pharmacometrics and Drug-Development

Lead Research Organisation: University of Bristol
Department Name: Social Medicine

Abstract

In the development of new drugs, studies are conducted to compare the relative benefits of the drug at different doses with placebo and/or other active drugs (which may also be at different doses). Furthermore, the health outcomes may be measured repeatedly over time. In order to decide whether to take the new drug forward into larger clinical trials, the results from all studies that have been conducted on a new drug are combined in meta-analysis to obtain a pooled estimate of the effect of the drug against placebo or active comparator drugs. Recently methods have been developed to allow for relative benefits to depend on dose and time of measurement in meta-analysis that compares the new drug with placebo (or another drug). However, there may be more than one comparator drug, and they have been measured at various different doses and times. Network meta-analysis is a technique that allows one to compare the relative benefits of multiple drugs that have been compared in randomised clinical trials, where not all drugs have been included in every study. This study aims to combine models of the relationships for the relative health benefits with dose and time, with network meta-analysis. This will allow us to combine information from studies comparing different drugs at different doses and different times, even though those studies may not have included the same dose and times.

Decisions as to which drugs to take forward into clinical trials, has substantial impact on all patients. Drug companies have limited resources, and so the decision to invest in one promising drug may come at the expense of another. It is therefore important to make drug-development decisions based on as much available evidence as possible. The methods developed in this project will allow as much existing evidence from comparative studies as possible to contribute to drug development decisions. Furthermore, we will explore the possibility of also incorporating evidence from studies that only sudy a single drug, or studies that compare drugs that we are not directly interested in, but that could help us understand the form of the relationships over dose and time.

The methods we will develop may require some strong assumptions. It is therefore very important to check whether those assumptions hold, and a key part of this work will be to look at methods to check assumptions and to check how well the models developed fit to the observed data. Decisions should be based on the most robust model predictions, and sensitivity to any assumptions explored.

This project will be a collaboration with project partner Pfizer, who will provide the datasets and expertise in dose and time course modelling. The University of Bristol team brings expertise in network meta-analysis, assessing model fit and consistency, and statistical computing. The collaboration is designed to ensure that the methods developed will be relevant to the needs of drug-development organisations, and the interaction with Pfizer will allow the methods to be used by that organisation, and publications and disemination plans will introduce the methods more widely. This approach will help the methods be used by industry to better invest their resources into drugs to improve patient health based on a better summary of the available evidence.

Technical Summary

Early phase trials compare drugs at different doses with placebo and/or other active drugs, often with repeated outcome measures over time. Drug development decisions are often informed by a meta-analysis of such studies. Recently methods have been developed to incorporate relationships with dose and time, termed Model-Based Meta-Analysis (MBMA). However, there may be more than one comparator drug, and the comparators may also have been measured at various different doses and times. Network meta-analysis is a technique that allows relative effects to be estimated across multiple treatments that have been compared in trials that form a connected network of treatment comparisons. Dose can be incorporated as a separate treatment, but this may lead to sparsely populated networks. This study aims to incorporate models for dose and time within a network meta-analysis framework. This will allow us to combine information from studies comparing different drugs at different doses and different times, even though those studies may not have included the same dose and times. Furthermore, we will explore the possibility of also incorporating evidence from single arm studies, and/or studies that compare drugs that we are not directly interested in, but that could help us understand the form of the relationships over dose and time.

The models we develop may require some strong assumptions. It is therefore important to check whether those assumptions hold, and a key part of this work will be to look at methods to check model fit, validity of model assumptions, in particular checks for inconsistency. The methods developed will allow the synthesis of all the relevant available evidence, and assessment of the robustness of the model predictions, to better inform drug-development decisions.
The industry project partner Pfizer will ensure that the methods developed will be relevant to the needs of drug-development organisations, facilitating the impact of the research.

Planned Impact

The methodology to be developed is planned to have commercial value. The project partner (Pfizer) has identified specific areas of pharmacometrics and drug-development where there is a need for network meta-analysis methodology to be developed. The industry partner (Pfizer) will directly benefit from this research in the development of methods for unmet industry needs to inform the drug-development process. The researcher will spend 12 one-week visits at Pfizer where they will interact with researchers at Pfizer, and present the results from their research. This will directly disseminate the methods to those who are involved in the drug-development process, and can be used by Pfizer staff to inform future development decisions. The decision to invest in a new drug that later is approved by reimbursement agencies, brings a new treatment option to patients, with associated benefits in health related quality of life. Conversely, the decision not to invest in a particular drug, frees resources to invest instead in more promising drugs, which again if later approved by reimbursement agencies, are expected to lead to increases in health-related quality of life. Improvements in the methods that inform such decisions may lead to new beneficial drugs being made available to patients more quickly. The methods may also be used to identify whether further evidence is needed to be collected in future studies, before a decision on a particular drug can be made. The research methods will be made available to those working in drug-development during the project, however, impact on patients would be expected several years after the project due to the long duration of the drug-development process.

It is expected that the methodology developed will be of value to other pharmaceutical companies, as well as Pfizer. The research will be published in peer reviewed journals, and presented at conferences, targeting statisticians in the pharmaceutical industry. The impact is therefore expected to be realised across the pharmaceutical industry, and not just the project partner.

The methodology is not restricted to early development. It is frequently the case in meta-analysis and network meta-analysis, that treatments/interventions differ in dose/intensity, and that lumping over dose/intensity can be the cause of heterogeneity. The methods developed here, may therefore have implications more generally amongst statisticians and health economists working in meta-analysis and network meta-analysis.

Publications

10 25 50
 
Description Evidence synthesis methods review for NICE Methods for Technology Appraisals methods update
Geographic Reach National 
Policy Influence Type Citation in other policy documents
Impact NICE recently updated their Guide to the Methods of Technology Appraisal which sets out methods to be followed by pharmaceutical and other health technology companies when making submissions for reimbursement on the NHS. I led a review of evidence synthesis methods which informed the evidence synthesis methods recommended in the NICE methods guide update. This has impact on the decisions made by NICE technology appraisal committees as to whether new health technologies represent value for money for the NHS, and therefore the impact is on both costs and quality of life.
URL https://www.nice.org.uk/process/pmg36/chapter/introduction-to-health-technology-evaluation
 
Description Member of NICE Guideline manual update expert panel
Geographic Reach National 
Policy Influence Type Membership of a guideline committee
URL https://www.nice.org.uk/process/pmg20/chapter/update-information
 
Description Member of NICE methods guide update task and finish group
Geographic Reach National 
Policy Influence Type Membership of a guideline committee
Impact The NICE methods guide sets out the methods to be used in submissions by health technology manufacturers to NICE for reimbursement in England. This committee contributed to the update of the methods guide which will be followed from 2022.
URL https://www.nice.org.uk/process/pmg36/chapter/introduction-to-health-technology-evaluation
 
Description Member of the NICE special committee on Models for the evaluation and purchase of antimicrobials project 2021
Geographic Reach National 
Policy Influence Type Membership of a guideline committee
 
Description Models for the evaluation and purchase of antimicrobials project
Geographic Reach National 
Policy Influence Type Membership of a guideline committee
Impact The outcome of the appraisals is currently confidential, but will impact on treatments available for resistant infections.
URL https://www.nice.org.uk/about/what-we-do/life-sciences/scientific-advice/models-for-the-evaluation-a...
 
Description Training for Pharmacometricians in Model Based Network Meta-Analysis
Geographic Reach Multiple continents/international 
Policy Influence Type Influenced training of practitioners or researchers
 
Description 20/39 Research Proposals: Production of Technology Assessment Reviews (TARs) for the National Institute for Health Research (NIHR)
Amount £4,325,000 (GBP)
Funding ID 20/39 
Organisation National Institute for Health Research 
Sector Public
Country United Kingdom
Start 04/2022 
End 03/2027
 
Title R package for modelling dose-response relationships in Network Meta-Analysis 
Description R package for modelling dose-response relationships in Network Meta-Analysis. Written and maintained by Hugo Pedder. https://CRAN.R-project.org/package=MBNMAdose 
Type Of Material Data analysis technique 
Year Produced 2020 
Provided To Others? Yes  
Impact Freely available R package to conduct the methods of analyses developed on this award. 
URL https://CRAN.R-project.org/package=MBNMAdose
 
Title R package for modelling time-course in Network Meta-Analysis (time MBNMA) 
Description R package for modelling time-course in Network Meta-Analysis (time MBNMA). Written and maintained by Hugo Pedder: https://CRAN.R-project.org/package=MBNMAtime 
Type Of Material Data analysis technique 
Year Produced 2020 
Provided To Others? Yes  
Impact Freely available R package to conduct the analyses developed on this award 
URL https://CRAN.R-project.org/package=MBNMAtime
 
Description Jose Lopez-Lopez MULTIMER project 
Organisation University of Murcia, Spain
Country Spain 
Sector Academic/University 
PI Contribution Expertise in component network meta-analysis
Collaborator Contribution Research project funded by the Spanish government PI Jose Lopez-Lopez who is conducting the research.
Impact No outputs yet
Start Year 2020
 
Description Model Based Network Meta-Analysis 
Organisation Pfizer Ltd
Country United Kingdom 
Sector Private 
PI Contribution Successful MRC grant, that is just about to begin. Pfizer bring model based meta-analysis expertise, and Bristol bring network meta-analysis skills. Funds come jointly from MRC MICA Methodology Research grant and from pfizer.
Collaborator Contribution Pfizer bring model based meta-analysis expertise and and access to datasets and relevant examples.
Impact MRC MICA grant
Start Year 2014
 
Description NICE Decision Support Unit 
Organisation University of Sheffield
Country United Kingdom 
Sector Academic/University 
PI Contribution As collaborators of the NICE Decision Support Unit we provide training, and technical support for evidence synthesis methods used in health technology assessments for NICE. Delivering short training courses for the Association of British Pharmaceutical Industry. Providing a review of evidence synthesis methods. Responding to queries relating to evidence synthesis methodology. Writing technical support documents for manufacturers making submissions to NICE.
Collaborator Contribution In exchange NICE Decision Support Unit staff provide ad hoc consultancy for the NICE Technical Support Unit that we run in Bristol.
Impact Welton, N.J., Phillippo, D.M., Owen, R., Jones, H.J., Dias, S., Bujkiewicz, S., Ades, A.E., Abrams, K.R. DSU Report. CHTE2020 Sources and Synthesis of Evidence; Update to Evidence Synthesis Methods. March 2020. https://nicedsu.sites.sheffield.ac.uk/methods-development/chte2020-sources-and-synthesis-of-evidence Dias S, Welton NJ, Sutton AJ, Ades AE. Introduction to evidence synthesis for decision making. NICE Decision Support Unit Evidence Synthesis Technical Support Document 1. April 2011. http://nicedsu.org.uk/technical-support-documents/evidence-synthesis-tsd-series/ Dias S, Welton NJ, Sutton AJ, Ades AE. A generalised linear modelling framework for pairwise and network meta-analysis of randomised controlled trials. NICE Decision Support Unit Evidence Synthesis Technical Support Document 2. August 2011. http://nicedsu.org.uk/technical-support-documents/evidence-synthesis-tsd-series/ Dias S, Sutton AJ, Welton NJ, Ades AE. Heterogeneity: subgroups, meta-regression, bias and bias-adjustment. NICE Decision Support Unit Evidence Synthesis Technical Support Document 3. September 2011. http://nicedsu.org.uk/technical-support-documents/evidence-synthesis-tsd-series/ Dias S, Welton NJ, Sutton AJ, Caldwell DM, Lu G, Ades AE. Inconsistency in networks of evidence based on randomised controlled trials. NICE Decision Support Unit Evidence Synthesis Technical Support Document 4. May 2011. http://nicedsu.org.uk/technical-support-documents/evidence-synthesis-tsd-series/ Dias S, Welton NJ, Sutton AJ, Ades AE. Evidence synthesis in the baseline natural history model. NICE Decision Support Unit Evidence Synthesis Technical Support Document 5. August 2011. http://nicedsu.org.uk/technical-support-documents/evidence-synthesis-tsd-series/ Dias S, Sutton AJ, Welton NJ, Ades AE. Embedding evidence synthesis in probabilistic cost-effectiveness analysis: software choices. NICE Decision Support Unit Evidence Synthesis Technical Support Document 6. May 2011. http://nicedsu.org.uk/technical-support-documents/evidence-synthesis-tsd-series/ Ades AE, Caldwell DM, Reken S, Welton NJ, Sutton AJ, Dias S. Evidence synthesis of treatment efficacy in decision making: a reviewer's checklist. NICE Decision Support Unit Evidence Synthesis Technical Support Document 7. January 2012. http://nicedsu.org.uk/technical-support-documents/evidence-synthesis-tsd-series/ Phillippo DM, Ades AE, Dias S, Palmer S, Abrams K, Welton NJ. Methods for population-adjusted indirect comparisons in submissions to NICE. NICE Decision Support Unit Technical Support Document 18. December 2016. http://nicedsu.org.uk/technical-support-documents/population-adjusted-indirect-comparisons-maic-and-stc/ Welton, N.J., Phillippo, D.M., Owen, R., Jones, H.J., Dias, S., Bujkiewicz, S., Ades, A.E., Abrams, K.R. DSU Report. CHTE2020 Sources and Synthesis of Evidence; Update to Evidence Synthesis Methods. March 2020. http://nicedsu.org.uk/wp-content/uploads/2020/11/CHTE-2020_final_20April2020_final.pdf
Start Year 2010
 
Description NICE Technical Support Unit 
Organisation National Institute for Health and Care Excellence (NICE)
Country United Kingdom 
Sector Public 
PI Contribution Provide training and technical support to NICE guideline developers, assist with research publications and develop methods to better support guideline developers.
Collaborator Contribution Financial support for staff. Disseminating methods into practise.
Impact Daly C, Dias S, Welton NJ, Anwer S, Ades AE. Meta-analysis. NICE Guidelines Technical Support Unit Guideline Methodology Document 1. January 2021. http://www.bristol.ac.uk/population-health-sciences/centres/cresyda/mpes/nice/guideline-methodology-documents-gmds/ Daly C, Welton NJ, Dias S, Anwer S, Ades AE. Meta-analysis of continuous outcomes. NICE Guidelines Technical Support Unit Guideline Methodology Document 2. January 2021. http://www.bristol.ac.uk/population-health-sciences/centres/cresyda/mpes/nice/guideline-methodology-documents-gmds/ Daly C, Anwer S, Welton NJ, Dias S, Ades AE. Meta-analysis of event outcomes. NICE Guidelines Technical Support Unit Guideline Methodology Document 3. January 2021. http://www.bristol.ac.uk/population-health-sciences/centres/cresyda/mpes/nice/guideline-methodology-documents-gmds/
Start Year 2011
 
Title MBNMAdose: An R package for running dose-response Model-Based Network Meta-Analysis models 
Description Fits Bayesian dose-response model-based network meta-analysis (MBNMA) that incorporate multiple doses within an agent by modelling different dose-response functions, as described by Mawdsley et al. (2016) . By modelling dose-response relationships this can connect networks of evidence that might otherwise be disconnected, and can improve precision on treatment estimates. Several common dose-response functions are provided; others may be added by the user. Various characteristics and assumptions can be flexibly added to the models, such as shared class effects. The consistency of direct and indirect evidence in the network can be assessed using unrelated mean effects models and/or by node-splitting at the treatment level. 
Type Of Technology Software 
Year Produced 2019 
Open Source License? Yes  
Impact The package has been used to investigate fitting dose-response MBNMA models in an analysis by Eli Lily. It is also being used to fit models for other, ongoing pieces of research into MBNMA. 
URL https://cran.r-project.org/web/packages/MBNMAdose/index.html
 
Title MBNMAtime: An R package for running time-course Model-Based Network Meta-Analysis models 
Description Fits Bayesian time-course models for model-based network meta-analysis (MBNMA) and model-based meta -analysis (MBMA) that account for repeated measures over time within studies by applying different time-course functions, following the method of Pedder et al. (2019) . The method allows synthesis of studies with multiple follow-up measurements that can account for time-course for a single or multiple treatment comparisons. Several general time-course functions are provided; others may be added by the user. Various characteristics can be flexibly added to the models, such as correlation between time points and shared class effects. The consistency of direct and indirect evidence in the network can be assessed using unrelated mean effects models and/or by node-splitting. 
Type Of Technology Software 
Year Produced 2019 
Open Source License? Yes  
Impact The software has been used to investigate fitting time-course MBNMA models to a dataset in a NICE guideline. 
URL https://cran.r-project.org/web/packages/MBNMAtime/index.html
 
Description Bayes Pharma. Model Based Network Meta-Analysis: A framework for evidence synthesis of dose-response models in randomised controlled trials. 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Presented research at the BayesPharma meeting which is primarily a group of Bayesian statisticians working in the pharmaceutical industry plus some delegates in academia. This is a key way to disseminate our research directly to those who may use it in industry.
Year(s) Of Engagement Activity 2016
 
Description Model Based Network Meta-Analysis for Decision-Making at PKUK 2015 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Invited speaker on Model Based Network Meta-Analysis for Decision-Making at the 2015 PKUK meeting of pharmacokinetics modelling in the pharmaceutical industry
Year(s) Of Engagement Activity 2015
 
Description PSI Conference: Promoting Statistical Insight and Collaboration in Drug Development. Model Based Network Meta-Analysis: A framework for evidence synthesis of dose-response models in randomised controlled trials. 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Presented research at the Statisticians in the Pharmaceutical Industry (PSI) meeting. This allowed us to diseminate research findings to those working in the pharmaceutical industry to help achieve impact.
Year(s) Of Engagement Activity 2016
 
Description Talk at Statisticians in Pharmaceutical Industry (PSI) Event: Model-Based Network Meta-Analysis in Pharmacometrics. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact Talk at the Statisticians in the Pharmaceutical Industry Event on Methods in drug development.
Year(s) Of Engagement Activity 2018
 
Description Training for the All Wales Therapeutics and Toxicology Centre 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Policymakers/politicians
Results and Impact Two half-day workshops on "Introduction to and Critical Appraisal of Indirect Comparisons and NMA". The training session was very hand-on with lots of practical discussion exercises and interaction with the audience. The methods covered are of direct relevance to the healthcare policy decisions that AWTCC make.
Year(s) Of Engagement Activity 2019
 
Description Workshop for Department of Health Research, India and HTAIn 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Policymakers/politicians
Results and Impact Training workshop introducing methods for evidence synthesis and indirect comparisons to those involved in health technology assessments to inform healthcare policy in India. I received positive feedback from attendees.
Year(s) Of Engagement Activity 2021
 
Description Workshop on Network Meta-Analysis to Pfizer 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Gave workshop on network meta-analysis in-house to Pfizer
Year(s) Of Engagement Activity 2015