MICA: Model Based Network Meta-Analysis for Pharmacometrics and Drug-Development
Lead Research Organisation:
University of Bristol
Department Name: Social Medicine
Abstract
In the development of new drugs, studies are conducted to compare the relative benefits of the drug at different doses with placebo and/or other active drugs (which may also be at different doses). Furthermore, the health outcomes may be measured repeatedly over time. In order to decide whether to take the new drug forward into larger clinical trials, the results from all studies that have been conducted on a new drug are combined in meta-analysis to obtain a pooled estimate of the effect of the drug against placebo or active comparator drugs. Recently methods have been developed to allow for relative benefits to depend on dose and time of measurement in meta-analysis that compares the new drug with placebo (or another drug). However, there may be more than one comparator drug, and they have been measured at various different doses and times. Network meta-analysis is a technique that allows one to compare the relative benefits of multiple drugs that have been compared in randomised clinical trials, where not all drugs have been included in every study. This study aims to combine models of the relationships for the relative health benefits with dose and time, with network meta-analysis. This will allow us to combine information from studies comparing different drugs at different doses and different times, even though those studies may not have included the same dose and times.
Decisions as to which drugs to take forward into clinical trials, has substantial impact on all patients. Drug companies have limited resources, and so the decision to invest in one promising drug may come at the expense of another. It is therefore important to make drug-development decisions based on as much available evidence as possible. The methods developed in this project will allow as much existing evidence from comparative studies as possible to contribute to drug development decisions. Furthermore, we will explore the possibility of also incorporating evidence from studies that only sudy a single drug, or studies that compare drugs that we are not directly interested in, but that could help us understand the form of the relationships over dose and time.
The methods we will develop may require some strong assumptions. It is therefore very important to check whether those assumptions hold, and a key part of this work will be to look at methods to check assumptions and to check how well the models developed fit to the observed data. Decisions should be based on the most robust model predictions, and sensitivity to any assumptions explored.
This project will be a collaboration with project partner Pfizer, who will provide the datasets and expertise in dose and time course modelling. The University of Bristol team brings expertise in network meta-analysis, assessing model fit and consistency, and statistical computing. The collaboration is designed to ensure that the methods developed will be relevant to the needs of drug-development organisations, and the interaction with Pfizer will allow the methods to be used by that organisation, and publications and disemination plans will introduce the methods more widely. This approach will help the methods be used by industry to better invest their resources into drugs to improve patient health based on a better summary of the available evidence.
Decisions as to which drugs to take forward into clinical trials, has substantial impact on all patients. Drug companies have limited resources, and so the decision to invest in one promising drug may come at the expense of another. It is therefore important to make drug-development decisions based on as much available evidence as possible. The methods developed in this project will allow as much existing evidence from comparative studies as possible to contribute to drug development decisions. Furthermore, we will explore the possibility of also incorporating evidence from studies that only sudy a single drug, or studies that compare drugs that we are not directly interested in, but that could help us understand the form of the relationships over dose and time.
The methods we will develop may require some strong assumptions. It is therefore very important to check whether those assumptions hold, and a key part of this work will be to look at methods to check assumptions and to check how well the models developed fit to the observed data. Decisions should be based on the most robust model predictions, and sensitivity to any assumptions explored.
This project will be a collaboration with project partner Pfizer, who will provide the datasets and expertise in dose and time course modelling. The University of Bristol team brings expertise in network meta-analysis, assessing model fit and consistency, and statistical computing. The collaboration is designed to ensure that the methods developed will be relevant to the needs of drug-development organisations, and the interaction with Pfizer will allow the methods to be used by that organisation, and publications and disemination plans will introduce the methods more widely. This approach will help the methods be used by industry to better invest their resources into drugs to improve patient health based on a better summary of the available evidence.
Technical Summary
Early phase trials compare drugs at different doses with placebo and/or other active drugs, often with repeated outcome measures over time. Drug development decisions are often informed by a meta-analysis of such studies. Recently methods have been developed to incorporate relationships with dose and time, termed Model-Based Meta-Analysis (MBMA). However, there may be more than one comparator drug, and the comparators may also have been measured at various different doses and times. Network meta-analysis is a technique that allows relative effects to be estimated across multiple treatments that have been compared in trials that form a connected network of treatment comparisons. Dose can be incorporated as a separate treatment, but this may lead to sparsely populated networks. This study aims to incorporate models for dose and time within a network meta-analysis framework. This will allow us to combine information from studies comparing different drugs at different doses and different times, even though those studies may not have included the same dose and times. Furthermore, we will explore the possibility of also incorporating evidence from single arm studies, and/or studies that compare drugs that we are not directly interested in, but that could help us understand the form of the relationships over dose and time.
The models we develop may require some strong assumptions. It is therefore important to check whether those assumptions hold, and a key part of this work will be to look at methods to check model fit, validity of model assumptions, in particular checks for inconsistency. The methods developed will allow the synthesis of all the relevant available evidence, and assessment of the robustness of the model predictions, to better inform drug-development decisions.
The industry project partner Pfizer will ensure that the methods developed will be relevant to the needs of drug-development organisations, facilitating the impact of the research.
The models we develop may require some strong assumptions. It is therefore important to check whether those assumptions hold, and a key part of this work will be to look at methods to check model fit, validity of model assumptions, in particular checks for inconsistency. The methods developed will allow the synthesis of all the relevant available evidence, and assessment of the robustness of the model predictions, to better inform drug-development decisions.
The industry project partner Pfizer will ensure that the methods developed will be relevant to the needs of drug-development organisations, facilitating the impact of the research.
Planned Impact
The methodology to be developed is planned to have commercial value. The project partner (Pfizer) has identified specific areas of pharmacometrics and drug-development where there is a need for network meta-analysis methodology to be developed. The industry partner (Pfizer) will directly benefit from this research in the development of methods for unmet industry needs to inform the drug-development process. The researcher will spend 12 one-week visits at Pfizer where they will interact with researchers at Pfizer, and present the results from their research. This will directly disseminate the methods to those who are involved in the drug-development process, and can be used by Pfizer staff to inform future development decisions. The decision to invest in a new drug that later is approved by reimbursement agencies, brings a new treatment option to patients, with associated benefits in health related quality of life. Conversely, the decision not to invest in a particular drug, frees resources to invest instead in more promising drugs, which again if later approved by reimbursement agencies, are expected to lead to increases in health-related quality of life. Improvements in the methods that inform such decisions may lead to new beneficial drugs being made available to patients more quickly. The methods may also be used to identify whether further evidence is needed to be collected in future studies, before a decision on a particular drug can be made. The research methods will be made available to those working in drug-development during the project, however, impact on patients would be expected several years after the project due to the long duration of the drug-development process.
It is expected that the methodology developed will be of value to other pharmaceutical companies, as well as Pfizer. The research will be published in peer reviewed journals, and presented at conferences, targeting statisticians in the pharmaceutical industry. The impact is therefore expected to be realised across the pharmaceutical industry, and not just the project partner.
The methodology is not restricted to early development. It is frequently the case in meta-analysis and network meta-analysis, that treatments/interventions differ in dose/intensity, and that lumping over dose/intensity can be the cause of heterogeneity. The methods developed here, may therefore have implications more generally amongst statisticians and health economists working in meta-analysis and network meta-analysis.
It is expected that the methodology developed will be of value to other pharmaceutical companies, as well as Pfizer. The research will be published in peer reviewed journals, and presented at conferences, targeting statisticians in the pharmaceutical industry. The impact is therefore expected to be realised across the pharmaceutical industry, and not just the project partner.
The methodology is not restricted to early development. It is frequently the case in meta-analysis and network meta-analysis, that treatments/interventions differ in dose/intensity, and that lumping over dose/intensity can be the cause of heterogeneity. The methods developed here, may therefore have implications more generally amongst statisticians and health economists working in meta-analysis and network meta-analysis.
Publications
Freeman SC
(2023)
Data visualisation approaches for component network meta-analysis: visualising the data structure.
in BMC medical research methodology
Gallardo-Gómez D
(2023)
Optimal dose and type of physical activity to improve functional capacity and minimise adverse events in acutely hospitalised older adults: a systematic review with dose-response network meta-analysis of randomised controlled trials
in British Journal of Sports Medicine
Pedder H
(2022)
Methods to assess evidence consistency in dose-response model based network meta-analysis.
in Statistics in medicine
Mavranezouli I
(2022)
A systematic review and network meta-analysis of topical pharmacological, oral pharmacological, physical and combined treatments for acne vulgaris.
in The British journal of dermatology
Mavranezouli I
(2022)
Cost-effectiveness of topical pharmacological, oral pharmacological, physical and combined treatments for acne vulgaris.
in Clinical and experimental dermatology
Van Der Vlist AC
(2021)
Which treatment is most effective for patients with Achilles tendinopathy? A living systematic review with network meta-analysis of 29 randomised controlled trials.
in British journal of sports medicine
Saueressig T
(2021)
Six Meta-analyses on Treatments for Femoroacetabular Impingement Syndrome in a Year and Readers Are None the Wiser: Methods Advice for Researchers Planning Meta-analysis of Data From Fewer Than 5 Trials.
in The Journal of orthopaedic and sports physical therapy
Bauer-Staeb C
(2021)
Effective dose 50 method as the minimal clinically important difference: Evidence from depression trials.
in Journal of clinical epidemiology
Vedhara K
(2021)
Effects of non-pharmacological interventions as vaccine adjuvants in humans: a systematic review and network meta-analysis.
in Health psychology review
Description | Evidence synthesis methods review for NICE Methods for Technology Appraisals methods update |
Geographic Reach | National |
Policy Influence Type | Citation in other policy documents |
Impact | NICE recently updated their Guide to the Methods of Technology Appraisal which sets out methods to be followed by pharmaceutical and other health technology companies when making submissions for reimbursement on the NHS. I led a review of evidence synthesis methods which informed the evidence synthesis methods recommended in the NICE methods guide update. This has impact on the decisions made by NICE technology appraisal committees as to whether new health technologies represent value for money for the NHS, and therefore the impact is on both costs and quality of life. |
URL | https://www.nice.org.uk/process/pmg36/chapter/introduction-to-health-technology-evaluation |
Description | Member of NICE Guideline manual update expert panel |
Geographic Reach | National |
Policy Influence Type | Membership of a guideline committee |
URL | https://www.nice.org.uk/process/pmg20/chapter/update-information |
Description | Member of NICE methods guide update task and finish group |
Geographic Reach | National |
Policy Influence Type | Membership of a guideline committee |
Impact | The NICE methods guide sets out the methods to be used in submissions by health technology manufacturers to NICE for reimbursement in England. This committee contributed to the update of the methods guide which will be followed from 2022. |
URL | https://www.nice.org.uk/process/pmg36/chapter/introduction-to-health-technology-evaluation |
Description | Member of the NICE special committee on Models for the evaluation and purchase of antimicrobials project 2021 |
Geographic Reach | National |
Policy Influence Type | Membership of a guideline committee |
Description | Models for the evaluation and purchase of antimicrobials project |
Geographic Reach | National |
Policy Influence Type | Membership of a guideline committee |
Impact | The outcome of the appraisals is currently confidential, but will impact on treatments available for resistant infections. |
URL | https://www.nice.org.uk/about/what-we-do/life-sciences/scientific-advice/models-for-the-evaluation-a... |
Description | Training for Pharmacometricians in Model Based Network Meta-Analysis |
Geographic Reach | Multiple continents/international |
Policy Influence Type | Influenced training of practitioners or researchers |
Description | 20/39 Research Proposals: Production of Technology Assessment Reviews (TARs) for the National Institute for Health Research (NIHR) |
Amount | £4,325,000 (GBP) |
Funding ID | 20/39 |
Organisation | National Institute for Health Research |
Sector | Public |
Country | United Kingdom |
Start | 04/2022 |
End | 03/2027 |
Title | R package for modelling dose-response relationships in Network Meta-Analysis |
Description | R package for modelling dose-response relationships in Network Meta-Analysis. Written and maintained by Hugo Pedder. https://CRAN.R-project.org/package=MBNMAdose |
Type Of Material | Data analysis technique |
Year Produced | 2020 |
Provided To Others? | Yes |
Impact | Freely available R package to conduct the methods of analyses developed on this award. |
URL | https://CRAN.R-project.org/package=MBNMAdose |
Title | R package for modelling time-course in Network Meta-Analysis (time MBNMA) |
Description | R package for modelling time-course in Network Meta-Analysis (time MBNMA). Written and maintained by Hugo Pedder: https://CRAN.R-project.org/package=MBNMAtime |
Type Of Material | Data analysis technique |
Year Produced | 2020 |
Provided To Others? | Yes |
Impact | Freely available R package to conduct the analyses developed on this award |
URL | https://CRAN.R-project.org/package=MBNMAtime |
Description | Jose Lopez-Lopez MULTIMER project |
Organisation | University of Murcia, Spain |
Country | Spain |
Sector | Academic/University |
PI Contribution | Expertise in component network meta-analysis |
Collaborator Contribution | Research project funded by the Spanish government PI Jose Lopez-Lopez who is conducting the research. |
Impact | No outputs yet |
Start Year | 2020 |
Description | Model Based Network Meta-Analysis |
Organisation | Pfizer Ltd |
Country | United Kingdom |
Sector | Private |
PI Contribution | Successful MRC grant, that is just about to begin. Pfizer bring model based meta-analysis expertise, and Bristol bring network meta-analysis skills. Funds come jointly from MRC MICA Methodology Research grant and from pfizer. |
Collaborator Contribution | Pfizer bring model based meta-analysis expertise and and access to datasets and relevant examples. |
Impact | MRC MICA grant |
Start Year | 2014 |
Description | NICE Decision Support Unit |
Organisation | University of Sheffield |
Country | United Kingdom |
Sector | Academic/University |
PI Contribution | As collaborators of the NICE Decision Support Unit we provide training, and technical support for evidence synthesis methods used in health technology assessments for NICE. Delivering short training courses for the Association of British Pharmaceutical Industry. Providing a review of evidence synthesis methods. Responding to queries relating to evidence synthesis methodology. Writing technical support documents for manufacturers making submissions to NICE. |
Collaborator Contribution | In exchange NICE Decision Support Unit staff provide ad hoc consultancy for the NICE Technical Support Unit that we run in Bristol. |
Impact | Welton, N.J., Phillippo, D.M., Owen, R., Jones, H.J., Dias, S., Bujkiewicz, S., Ades, A.E., Abrams, K.R. DSU Report. CHTE2020 Sources and Synthesis of Evidence; Update to Evidence Synthesis Methods. March 2020. https://nicedsu.sites.sheffield.ac.uk/methods-development/chte2020-sources-and-synthesis-of-evidence Dias S, Welton NJ, Sutton AJ, Ades AE. Introduction to evidence synthesis for decision making. NICE Decision Support Unit Evidence Synthesis Technical Support Document 1. April 2011. http://nicedsu.org.uk/technical-support-documents/evidence-synthesis-tsd-series/ Dias S, Welton NJ, Sutton AJ, Ades AE. A generalised linear modelling framework for pairwise and network meta-analysis of randomised controlled trials. NICE Decision Support Unit Evidence Synthesis Technical Support Document 2. August 2011. http://nicedsu.org.uk/technical-support-documents/evidence-synthesis-tsd-series/ Dias S, Sutton AJ, Welton NJ, Ades AE. Heterogeneity: subgroups, meta-regression, bias and bias-adjustment. NICE Decision Support Unit Evidence Synthesis Technical Support Document 3. September 2011. http://nicedsu.org.uk/technical-support-documents/evidence-synthesis-tsd-series/ Dias S, Welton NJ, Sutton AJ, Caldwell DM, Lu G, Ades AE. Inconsistency in networks of evidence based on randomised controlled trials. NICE Decision Support Unit Evidence Synthesis Technical Support Document 4. May 2011. http://nicedsu.org.uk/technical-support-documents/evidence-synthesis-tsd-series/ Dias S, Welton NJ, Sutton AJ, Ades AE. Evidence synthesis in the baseline natural history model. NICE Decision Support Unit Evidence Synthesis Technical Support Document 5. August 2011. http://nicedsu.org.uk/technical-support-documents/evidence-synthesis-tsd-series/ Dias S, Sutton AJ, Welton NJ, Ades AE. Embedding evidence synthesis in probabilistic cost-effectiveness analysis: software choices. NICE Decision Support Unit Evidence Synthesis Technical Support Document 6. May 2011. http://nicedsu.org.uk/technical-support-documents/evidence-synthesis-tsd-series/ Ades AE, Caldwell DM, Reken S, Welton NJ, Sutton AJ, Dias S. Evidence synthesis of treatment efficacy in decision making: a reviewer's checklist. NICE Decision Support Unit Evidence Synthesis Technical Support Document 7. January 2012. http://nicedsu.org.uk/technical-support-documents/evidence-synthesis-tsd-series/ Phillippo DM, Ades AE, Dias S, Palmer S, Abrams K, Welton NJ. Methods for population-adjusted indirect comparisons in submissions to NICE. NICE Decision Support Unit Technical Support Document 18. December 2016. http://nicedsu.org.uk/technical-support-documents/population-adjusted-indirect-comparisons-maic-and-stc/ Welton, N.J., Phillippo, D.M., Owen, R., Jones, H.J., Dias, S., Bujkiewicz, S., Ades, A.E., Abrams, K.R. DSU Report. CHTE2020 Sources and Synthesis of Evidence; Update to Evidence Synthesis Methods. March 2020. http://nicedsu.org.uk/wp-content/uploads/2020/11/CHTE-2020_final_20April2020_final.pdf |
Start Year | 2010 |
Description | NICE Technical Support Unit |
Organisation | National Institute for Health and Care Excellence (NICE) |
Country | United Kingdom |
Sector | Public |
PI Contribution | Provide training and technical support to NICE guideline developers, assist with research publications and develop methods to better support guideline developers. |
Collaborator Contribution | Financial support for staff. Disseminating methods into practise. |
Impact | Daly C, Dias S, Welton NJ, Anwer S, Ades AE. Meta-analysis. NICE Guidelines Technical Support Unit Guideline Methodology Document 1. January 2021. http://www.bristol.ac.uk/population-health-sciences/centres/cresyda/mpes/nice/guideline-methodology-documents-gmds/ Daly C, Welton NJ, Dias S, Anwer S, Ades AE. Meta-analysis of continuous outcomes. NICE Guidelines Technical Support Unit Guideline Methodology Document 2. January 2021. http://www.bristol.ac.uk/population-health-sciences/centres/cresyda/mpes/nice/guideline-methodology-documents-gmds/ Daly C, Anwer S, Welton NJ, Dias S, Ades AE. Meta-analysis of event outcomes. NICE Guidelines Technical Support Unit Guideline Methodology Document 3. January 2021. http://www.bristol.ac.uk/population-health-sciences/centres/cresyda/mpes/nice/guideline-methodology-documents-gmds/ |
Start Year | 2011 |
Title | MBNMAdose: An R package for running dose-response Model-Based Network Meta-Analysis models |
Description | Fits Bayesian dose-response model-based network meta-analysis (MBNMA) that incorporate multiple doses within an agent by modelling different dose-response functions, as described by Mawdsley et al. (2016) . By modelling dose-response relationships this can connect networks of evidence that might otherwise be disconnected, and can improve precision on treatment estimates. Several common dose-response functions are provided; others may be added by the user. Various characteristics and assumptions can be flexibly added to the models, such as shared class effects. The consistency of direct and indirect evidence in the network can be assessed using unrelated mean effects models and/or by node-splitting at the treatment level. |
Type Of Technology | Software |
Year Produced | 2019 |
Open Source License? | Yes |
Impact | The package has been used to investigate fitting dose-response MBNMA models in an analysis by Eli Lily. It is also being used to fit models for other, ongoing pieces of research into MBNMA. |
URL | https://cran.r-project.org/web/packages/MBNMAdose/index.html |
Title | MBNMAtime: An R package for running time-course Model-Based Network Meta-Analysis models |
Description | Fits Bayesian time-course models for model-based network meta-analysis (MBNMA) and model-based meta -analysis (MBMA) that account for repeated measures over time within studies by applying different time-course functions, following the method of Pedder et al. (2019) . The method allows synthesis of studies with multiple follow-up measurements that can account for time-course for a single or multiple treatment comparisons. Several general time-course functions are provided; others may be added by the user. Various characteristics can be flexibly added to the models, such as correlation between time points and shared class effects. The consistency of direct and indirect evidence in the network can be assessed using unrelated mean effects models and/or by node-splitting. |
Type Of Technology | Software |
Year Produced | 2019 |
Open Source License? | Yes |
Impact | The software has been used to investigate fitting time-course MBNMA models to a dataset in a NICE guideline. |
URL | https://cran.r-project.org/web/packages/MBNMAtime/index.html |
Description | Bayes Pharma. Model Based Network Meta-Analysis: A framework for evidence synthesis of dose-response models in randomised controlled trials. |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Industry/Business |
Results and Impact | Presented research at the BayesPharma meeting which is primarily a group of Bayesian statisticians working in the pharmaceutical industry plus some delegates in academia. This is a key way to disseminate our research directly to those who may use it in industry. |
Year(s) Of Engagement Activity | 2016 |
Description | Model Based Network Meta-Analysis for Decision-Making at PKUK 2015 |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Industry/Business |
Results and Impact | Invited speaker on Model Based Network Meta-Analysis for Decision-Making at the 2015 PKUK meeting of pharmacokinetics modelling in the pharmaceutical industry |
Year(s) Of Engagement Activity | 2015 |
Description | PSI Conference: Promoting Statistical Insight and Collaboration in Drug Development. Model Based Network Meta-Analysis: A framework for evidence synthesis of dose-response models in randomised controlled trials. |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Industry/Business |
Results and Impact | Presented research at the Statisticians in the Pharmaceutical Industry (PSI) meeting. This allowed us to diseminate research findings to those working in the pharmaceutical industry to help achieve impact. |
Year(s) Of Engagement Activity | 2016 |
Description | Talk at Statisticians in Pharmaceutical Industry (PSI) Event: Model-Based Network Meta-Analysis in Pharmacometrics. |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Industry/Business |
Results and Impact | Talk at the Statisticians in the Pharmaceutical Industry Event on Methods in drug development. |
Year(s) Of Engagement Activity | 2018 |
Description | Training for the All Wales Therapeutics and Toxicology Centre |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | Regional |
Primary Audience | Policymakers/politicians |
Results and Impact | Two half-day workshops on "Introduction to and Critical Appraisal of Indirect Comparisons and NMA". The training session was very hand-on with lots of practical discussion exercises and interaction with the audience. The methods covered are of direct relevance to the healthcare policy decisions that AWTCC make. |
Year(s) Of Engagement Activity | 2019 |
Description | Workshop for Department of Health Research, India and HTAIn |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Policymakers/politicians |
Results and Impact | Training workshop introducing methods for evidence synthesis and indirect comparisons to those involved in health technology assessments to inform healthcare policy in India. I received positive feedback from attendees. |
Year(s) Of Engagement Activity | 2021 |
Description | Workshop on Network Meta-Analysis to Pfizer |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Industry/Business |
Results and Impact | Gave workshop on network meta-analysis in-house to Pfizer |
Year(s) Of Engagement Activity | 2015 |