Investigation of discrepancies between direct and indirect comparison of competing healthcare interventions
Lead Research Organisation:
University of East Anglia
Department Name: Allied Health Professions
Abstract
To choose the best treatment from several available treatments to an illness, the best evidence is from randomised trials (RCTs) where different treatments have been directly tested against each other (direct comparison).
A statistical method known as adjusted indirect comparison may be helpful when available treatments have not been directly compared in RCTs. However, this indirect comparison method may produce results that are different from a direct comparison of different treatments.
We aim to investigate factors associated with the difference between direct and indirect approaches. We will see if indirect comparison gives the same answer as trials where the different treatments have been directly compared. If the two methods give different answers, we will look through the studies in detail, and try to work out what may have caused the difference.
The research we plan to do is very important because government bodies like the National Institute of Clinical Excellence may have to use indirect comparison methods in choosing treatments, when no direct comparison study is available. Our research findings should help us recognize situations where the indirect comparison may be a good, reliable choice, and also when it can give us wrong answers.
A statistical method known as adjusted indirect comparison may be helpful when available treatments have not been directly compared in RCTs. However, this indirect comparison method may produce results that are different from a direct comparison of different treatments.
We aim to investigate factors associated with the difference between direct and indirect approaches. We will see if indirect comparison gives the same answer as trials where the different treatments have been directly compared. If the two methods give different answers, we will look through the studies in detail, and try to work out what may have caused the difference.
The research we plan to do is very important because government bodies like the National Institute of Clinical Excellence may have to use indirect comparison methods in choosing treatments, when no direct comparison study is available. Our research findings should help us recognize situations where the indirect comparison may be a good, reliable choice, and also when it can give us wrong answers.
Technical Summary
BACKGROUND: Adjusted indirect comparison (AIC) has been increasingly used in the evaluation of healthcare interventions. Although the basic assumption for a valid AIC is theoretically clear, there is only limited empirical evidence on factors associated with the validity of AIC. Practically useful methods for assessing the appropriateness of AIC and mixed treatment comparison (MTC) have not been systematically developed and tested. Consequently, the use of AIC and MTC is still controversial and over-shadowed by concern of their misuse. We propose a methodological research to address these limitations and concerns.
OBJECTIVES: The proposed research aims (1) to greatly expand empirical evidence on the validity of AIC, (2) to develop and test methods for assessing clinical similarities in AIC, and for assessing clinical consistency between direct comparison (DC) and indirect comparison (AIC), and (3) to investigate factors associated with the discrepancy between direct and indirect estimates.
RESEARCH PLANS: We will search the Cochrane Library to identify Cochrane Systematic Reviews (CSRs) that provide sufficient data to compare competing interventions by both DC and AIC (using Bucher et al s method). The discrepancy between the DC and AIC will be measured by the difference between the two estimates. We will extract data on trial participants, interventions compared, outcome measures, results and other study characteristics from trials in relevant CSRs. Based on data compiled above, we will assess similarities of the two sets of trials involved in AIC, and assess clinical consistency between DC and AIC. The results of similarity and consistency assessment will be conveyed to a clinical similarity score and a clinical consistency score for each meta-analysis. Reliability and validity of the similarity and consistency score will be investigated. Using the observed discrepancy between DC and AIC estimates as the dependent variable, meta-regression analyses will be conducted to investigate factors (including similarity and consistency scores, types of interventions, number of trials, heterogeneity, etc) that may be associated with bias in AIC. Exploratory sensitivity analyses will be conducted when possible to improve trials comparability by excluding trials that cause poor clinical similarity or inconsistency, and by using meta-regression to adjust for imbalance in factors which modify treatment effects. We will investigate whether the improved clinical similarity and/or consistency is associated with reduced discrepancy.
APPLICATION OF THE RESEARCH RESULTS: Findings from the proposed research will enable us to provide practical recommendations for valid indirect and/or mixed treatment comparison in the evaluation of competing healthcare interventions.
OBJECTIVES: The proposed research aims (1) to greatly expand empirical evidence on the validity of AIC, (2) to develop and test methods for assessing clinical similarities in AIC, and for assessing clinical consistency between direct comparison (DC) and indirect comparison (AIC), and (3) to investigate factors associated with the discrepancy between direct and indirect estimates.
RESEARCH PLANS: We will search the Cochrane Library to identify Cochrane Systematic Reviews (CSRs) that provide sufficient data to compare competing interventions by both DC and AIC (using Bucher et al s method). The discrepancy between the DC and AIC will be measured by the difference between the two estimates. We will extract data on trial participants, interventions compared, outcome measures, results and other study characteristics from trials in relevant CSRs. Based on data compiled above, we will assess similarities of the two sets of trials involved in AIC, and assess clinical consistency between DC and AIC. The results of similarity and consistency assessment will be conveyed to a clinical similarity score and a clinical consistency score for each meta-analysis. Reliability and validity of the similarity and consistency score will be investigated. Using the observed discrepancy between DC and AIC estimates as the dependent variable, meta-regression analyses will be conducted to investigate factors (including similarity and consistency scores, types of interventions, number of trials, heterogeneity, etc) that may be associated with bias in AIC. Exploratory sensitivity analyses will be conducted when possible to improve trials comparability by excluding trials that cause poor clinical similarity or inconsistency, and by using meta-regression to adjust for imbalance in factors which modify treatment effects. We will investigate whether the improved clinical similarity and/or consistency is associated with reduced discrepancy.
APPLICATION OF THE RESEARCH RESULTS: Findings from the proposed research will enable us to provide practical recommendations for valid indirect and/or mixed treatment comparison in the evaluation of competing healthcare interventions.