Modern causal methods to estimate the impact of Individual and Group Health Policies using routinely collected data

Policy makers are interested in evaluating the causal effects and cost-effectiveness of health and care policies introduced in primary or secondary health or social care settings (GP practices, hospitals or residential/nursing homes). Such policies are often introduced without having first conducted a randomised study and therefore policy evaluation usually relies on retrospective observational data, usually routinely collected, such as electronic health records (EHRs) from GP practices and hospital trusts.
There are several challenges in using EHR data for policy evaluation, which need to be addressed to enable unbiased estimates of policy impacts. A key challenge is to account for the fact that those exposed to the policy may systematically differ from those unexposed in multiple respects. Variables simultaneously associated with the outcome of interest and the exposure are known as confounders. Failure to adjust for confounding can lead to substantial bias in estimates of policy effects. Other sources of bias when using EHRs stem from inaccurately defining the study populations, interventions of interest and time origins. The specific nature of the data also brings additional challenges: the large volume of data, often inconsistently recorded, increases the complexity of identifying confounders and effect moderators. In high-dimensional settings (i.e. with many covariates), modelling complex confounding patterns correctly, especially given their often time-dependent nature, is difficult. Misspecification of the relationships between an outcome, the exposure and identified confounders is also a serious concern, because causal inference inevitably requires out-of-sample extrapolation; misspecification may result in misleading conclusions that are difficult to diagnose.
In recent years, the field of 'causal inference' has provided concepts, tools and statistical analysis methods that facilitate estimation of causal effects from observational data. However, there remain a number of gaps in the toolkit. One such gap is that the literature has focused primarily on individuallevel interventions, rather than policy interventions at a group level. Furthermore, extensions of methods to accommodate the particular complexities of EHR data have only more recently started to emerge and have not yet come into wide use.
The proposed project is a collaboration between the LSHTM Department of Medical Statistics, and the Improvement Analytics Unit (IAU) hosted at The Health Foundation (THF). The IAU is a partnership between THF and NHS England and NHS Improvement, created to evaluate complex health care policies implemented at local to national level to support decision making as well as inform national policy. The IAU uses novel patient-level linked datasets, which include information on secondary care use, mortality, as well as socio-demographic information, where appropriate linked with data from primary care, social or community care, or other local services. The unit's work programme varies over time, but at present focusses on (1) the impact of Digital First Primary Care models on general practice and subsequent hospital use, (2) the long term impact of integrated care initiatives on secondary care outcomes, and (3) the impact of the COVID Oximetry at home programme on clinical effectiveness and COVID-19 mortality. The proposed PhD project will develop causal inference methodology for group-level policy interventions using EHR data, and apply this to evaluate the policy impact of these 3 projects. The resulting policy evaluation estimates will be more reliable in terms of acknowledging the groupstructure, accounting for high number of confounders and model uncertainty. The methods developed in this proposed research will equip future analysts with better tools for evidence based policy evaluation that will better support policy-makers and health services managers in their quest to deliver better-value high-quality care.