Multiple imputation methods for valid causal treatment effects estimation after departures from protocol

Lead Research Organisation: London School of Hygiene & Tropical Medicine
Department Name: Epidemiology and Population Health


Randomised controlled trials are considered the gold standard in determining the effect of an intervention. Two issues that often undermine the credibility of effectiveness measures are non-compliance with treatment allocation (participants who do not adhere to the protocol, e.g. by not receiving the intended treatment) and missing data (e.g. loss to follow-up).

Both these issues are relevant to the adoption of the Intention-to-treat (ITT) analysis, which aims to measure treatment effectiveness. The ITT principle states that all individuals randomised in a clinical trial should be included in the analysis, in the groups to which they were randomised, regardless of any departures from the randomised treatment. Following this principle preserves the benefits of randomisation, i.e. having treatment groups that do not differ systematically on any factors except those assigned in the trial. But such pragmatic estimates may not be the only estimates of interest, and given the considerable costs and high number of patients involved in a typical confirmatory clinical trial, efforts should be made to obtain valid explanatory treatment effects, i.e. the "real" treatment effects that a patient taking the treatment 100% as prescribed can expect on average.

Ad hoc methods which ignore treatment allocation may lead to incorrect estimates of treatment effect. Existing statistical methods can handle some complex longitudinal settings, e.g. when compliance with treatment varies with time and is associated with the clinical outcome of interest (assumed to be a continuous measure). However, they rely on assumptions which are difficult to understand and verify, and involve sophisticated numerical iterative procedures to obtain the estimates of interest. Even the most sophisticated method cannot currently handle the estimation of causal treatment effects on binary outcomes (outcomes that take only two values, e.g. mortality).
It is therefore important that practical and efficient methods are available for handling non-compliance and missing data, especially when the data structure is complex, in a unified, transparent, and systematic manner.

The proposed research aims to develop new statistical methods to deal with these two issues within a single unifying framework and under transparent assumptions.
This novel method is based on multiple imputation, which is a practical and flexible method already widely used to address missing data problems. The methods developed will enable researchers to obtain precise and unbiased estimates of the causal effect of treatment, thereby answering important clinical and public health questions about the average treatment effects at different levels of compliance. This may in turn help individual clinicians and patients take more informed decisions about which treatment to follow.

The methods developed will be published in scientific journals. In addition, new methods will be implemented into statistical software packages to encourage uptake.
The work will be conducted by the Fellow.

Technical Summary

The aim is to develop methods based on multiple imputation (MI) for handling deviations from protocol, defined as non-compliance and drop-outs, in time-varying complex randomised clinical trial data where current methods are not applicable (hierarchical data, binary outcomes with continuous non-compliance and bivariate outcomes).
This will be achieved by extending a novel method (called CRK henceforth), based on pattern-mixture models. It uses MI to handle drop-outs and non-compliance in a unified manner, and estimates efficacy (called de jure), by imputing post-deviation data from a distribution whose parameters have been estimated using the observed data from those subjects who comply in their corresponding allocated treatment arm.

Firstly, I will relate this approach to the existing causal inference methods. I will establish relationships between the different assumptions used by each method for parameter identification.

Secondly, a Principal Stratification (PS) approach that uses MI estimation for the complier-average causal effect (CACE) will be extended to time-varying binary non-compliance settings. PS divides the trial population into compliance classes. As the number of possible compliance histories grows exponentially with the number of follow-up visits, CACE estimation becomes high-dimensional. A latent-class representing the time-invariant super-compliance class is used as a pragmatic solution.

Thirdly, I will extend the CRK approach to handle time-varying non-compliance of bivariate outcomes (e.g. cost-effectiveness) and hierarchical data (e.g. data from a cluster randomised trial).

Finally, I will extend the method to binary outcomes in the time-varying non-compliance setting and develop a framework for sensitivity analysis to study the robustness of results when there are departures from assumptions.

Throughout, newly-developed methods will be implemented in statistical packages for R and STATA.

Planned Impact

The main beneficiaries of the research output will be researchers interested in testing new interventions through randomised clinical trials and in finding unbiased efficacy estimates. These include trial statisticians and health economists as well as clinicians, in academia and pharmaceutical companies, and those involved in the NIHR Health Technology Appraisal programme.

The research output will also help inform future clinical trial design, by highlighting the importance of good quality data collection of levels of compliance and reasons for non-compliance, in particular if related to adverse effects or to the clinical outcome of interest.

Other beneficiaries include public health policymakers. Reimbursement authorities require cost-effectiveness analysis to determine if a new treatment is cost-effective. Frequently, this is done using clinical trial data, which often fail to adjust for non-compliance, even though the levels of compliance in the trial population are likely to differ from those in general practice. Thus providing a tool for sensitivity of cost-effectiveness estimates to changes in compliance would help medical decision makers, such as NICE, in deciding the robustness of the results.

It will also have an impact on clinical decision making. Individual patients and clinicians may make more informed choices as to: (a) what benefit to expect from following a certain treatment to the letter, or (b) when a particular individual's compliance with treatment is predicted to be less than the "effective" threshold, the clinician and the individual may opt for a different treatment, whose rigours and side-effects are such, that the patient is expected to adhere sufficiently in order to benefit from it. Therefore, I envisage my research could go some way towards personalised medicine, an area of growing interest in the research medical community.


10 25 50
Description Bloomsbury Doctoral Research Centre PhD studentship
Amount £50,000 (GBP)
Organisation Economic and Social Research Council 
Sector Public
Country United Kingdom
Start 09/2015 
End 09/2018
Description Medical Research Council London Intercollegiate Doctorate
Amount £80,000 (GBP)
Organisation Medical Research Council (MRC) 
Sector Public
Country United Kingdom
Start 08/2018 
End 09/2022
Description Sir Henry Dale Fellowship
Amount £927,042 (GBP)
Funding ID 218554/Z/19/Z 
Organisation Sir Henry Dale Fellowships 
Sector Charity/Non Profit
Country United Kingdom
Start 04/2020 
End 04/2025
Description UK Wellcome Trust Institutional Strategic Support Fund-LSHTM Fellowship
Amount £85,000 (GBP)
Funding ID 204928/Z/16/Z 
Organisation London School of Hygiene and Tropical Medicine (LSHTM) 
Sector Academic/University
Country United Kingdom
Start 03/2018 
End 10/2019
Title Instrumental variable methods for cost-effectiveness causal estimates 
Description Developed statistical methods for obtaining causal effects of treatment in cost-effectiveness analysis, using instrumental variables techniques, by three-stage least squares and Bayesian methods. 
Type Of Material Improvements to research infrastructure 
Provided To Others? No  
Impact n/a 
Title Multiple imputation methods to adjust for treatment non-compliance in randomised trials 
Description I have made available a new strategy to using multiple imputation to perform complier-average causal estimates (CACE) for randomised trials with departures from protocol. Trialists often find pure instrumental variables approaches, and Bayesian approaches very complex, but feel comfortable with multiple imputation techniques, so higher uptake of this method is envisaged. 
Type Of Material Improvements to research infrastructure 
Year Produced 2017 
Provided To Others? Yes  
Impact The impact is not yet seen, as this is a recent development 
Description Training seminar for the Fit to Study group in Oxford 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact The Fit to Study Cluster randomised trial researchers were faced with questions about how to best adjust for noncompliance and deal with the missing data in their trial. I was invited to give a talk, and after it, we discussed ways in which the trial analysis can be conducted to obtain statistically valid results.
Year(s) Of Engagement Activity 2019
Description Visit to Clinical Trials Unit in Leeds University 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact I was invited to a half-day meeting with the main trial statisticians, and health economist at Leeds Clinial Trials Unit in Leeds.
They are interested in methods to obtain causal inference after non-adherence to randomised treatment has occurred in cluster randomised trails and cost-effectiveness studies alongside.
As a result of our discussions, I will become a "consultant" methodology statistician to their team, and will help them choose the correct method for their needs.
We are also planning on running a workshop on adherence-adjustment methods in the Autumn.
Year(s) Of Engagement Activity 2017