Improving statistical methods to address confounding in the economic evaluation of health interventions

Policy makers worldwide use health economic evaluation to help decide which health care interventions to provide. For the cost-effectiveness analysis of pharmaceuticals, randomised trials are seen as the "gold standard" source of evidence for economic evaluation, because randomisation ensures that any difference between the health and economic outcomes of the treated and control patients reflects the causal effect of the treatment.

In many settings, including the evaluation of new devices and of diagnostic tests, clinical guideline development, and the evaluation of new health policy initiatives, no relevant trial evidence is available. Such economic evaluations need to use non-randomised evidence, for example registry data, or cohort studies. For these studies, randomisation between treatment arms is not ensured any longer. Here, a simple comparison of treatment groups would yield selection bias in the estimated quantity of interest, due to confounding factors, i.e. patient characteristics that make it more likely for a patient to receive one treatment over the other, and also influence the health outcomes and eventual treatment costs. Selection bias due to confounding can be adjusted for if appropriate statistical methods are used. However, as a recent systematic review has demonstrated, in published economic evaluations, the quality of statistical analysis to address this problem is unsatisfactory. Results from such studies can lead to the wrong conclusions on cost-effectiveness, and to health care resources being misallocated.
Methods developments for addressing selection bias in economic evaluation so far has been limited to relatively simple settings, such as comparing two treatments, which do not change over time. However, these settings might not characterise more complex evaluations. First, decision makers may require information not just about the cost-effectiveness of a binary treatment, but also about what intensity of treatment to provide. For example, when introducing a new financial incentive, the policy maker may want to know what the optimal level of incentive is. Second, in clinical practice, treatment provided can respond to the patient's characteristics, for example cancer treatment is switched according to tumour progression. Third, many interventions, typically health policy initiatives, are implemented at the level of an institution (e.g. NHS trust) or for an entire geographical region (e.g. health authority), and adjusting for confounding might be challenging due to the lack of an appropriate control group.

Currently there is a lack of methodological guidance for these settings. This might lead to the use of inappropriate methods, resulting in severely biased estimates, or worse; completely discourage analysts and decision makers from exploiting non-randomised evidence for economic evaluation. Methods that can address confounding in these settings are at the forefront of developments in the causal inference literature, however these methods have yet to be translated, and potentially extended to the setting of economic evaluation.
I propose to conduct a comprehensive research programme to address this gap in knowledge, using both simulation work and data from clinical and policy areas of high relevance. The research will assess and if necessary, extend alternative methods from the causal inference literature for addressing confounding in economic evaluation. This research will enable me to provide recommendations on which methods are appropriate in an economic evaluation setting, towards applied researchers and decision makers. By thorough dissemination of the methods, this research aims to improve the quality of statistical analysis in economic evaluation, leading to more accurate cost-effectiveness results, and a stronger evidence for decision making. This research will therefore help ensure that scarce resources are allocated in the best ways for improving population health in the UK.

Motivation: Confounding is a major methodological challenge in health economic evaluations that use non-randomised studies (NRS). Currently recommended methods are not directly applicable for complex settings, such as when cost-effectiveness of a continuous or dynamic treatment regime is of interest, and when treatment is administered at the area level, without an appropriate control group. The causal inference literature proposes methods to address confounding in these settings, however to deal with the specific challenges of economic evaluation, these methods need to be critically assessed, and extended.
Methodology: First, I will compare the generalised propensity score method with regression approaches, to model dose-response relationships in economic evaluation. Second, I will compare novel causal inference methods for adjusting for time-varying confounding when evaluating dynamic treatment regimes, such as the targeted maximum likelihood estimation and parametric g-computation, when estimating parameters for decision analytical models. Third, in the evaluation of health policies without an appropriate control group, I will consider the synthetic control method that makes weaker assumptions about unobserved confounding than traditionally used difference-in-differences estimation, and extend it to handle health care data at the patient and provider level. I will then develop a new framework for assessing the robustness of cost-effectiveness estimates under alternative assumptions around unobserved confounders. Throughout, I will use case studies and simulation studies to compare new statistical methods to traditional approaches in typical settings for economic evaluation.

Research output: This project will provide insights about the relative merits of alternative methods for addressing confounding across complex settings in health economic evaluation. Methods will be widely disseminated so that future studies can provide a sound basis for policy making.

This research is anticipated to benefit a larger audience beyond the academic community, including those involved in health technology assessment and policy evaluation with NICE and the Department of Health (DH), members of the pharmaceutical and medical devices industry, and through these, the NHS and the wider public.
This project will improve the methods for addressing confounding in health economic evaluation, so that future studies can provide a sounder basis for policy making. To raise awareness of new methods available, I will disseminate directly to policy makers such as NICE. I will also disseminate results towards those who are involved in conducting cost-effectiveness analyses, including research scientists in the pharmaceutical and medical devices industry. The research will also affect those involved in designing observational studies, by highlighting the importance of collecting a rich source of patient and provider characteristics to facilitates credible statistical analysis. By improving the quality of economic evaluation studies, on the long term, this research will help ensure that scarce societal resources are allocated in the best ways to improve population health.
The analytic approaches developed will also contribute to improving the evaluation of new health policy initiatives, such as incentive schemes for health care providers. The research will engage members of the Policy Research Unit in Policy Innovation Research (PIRU), a DH-funded research unit. The director of PIRU, Prof Nick Mays is a collaborator and will facilitate communication of the findings of the proposed research directly to policy makers and analysts at DH. A better understanding of methodological challenges in the evaluation process is expected to lead to better design of health policy pilots and ultimately to better evidence on which health policies most benefit population health.
The activities undertaken to facilitate impact of the research to the beneficiaries identified here are detailed in the attachment "Pathways to Impact".