MICA: Clinical trial estimands: from definition to estimation

Randomised clinical trials represent the gold standard approach for testing whether new treatments for diseases work better than existing treatments and quantifying the magnitude of the benefit. In principle the analysis of such trials is simple - one compares the chosen outcome measure of patients in one group with the patients in the other group. In practice a number of complications may arise which make this comparison difficult to interpret or impossible even to calculate. One example is trials in which patients may change from the treatment that they were randomly assigned to receive during the follow-up period, either to the alternative treatment, no treatment at all, or they may start taking additional treatment(s). A second example is in trials which aim to compare (for example) cholesterol treatments in terms of their effects on death due to cardiovascular disease. This comparison is complicated by the fact that some patients may die of other causes, such as cancer. In a simple analysis comparing the number of patients who died due to cardiovascular disease between the two groups, a new treatment could for example reduce the chances of death due to cardiovascular disease, but only by virtue of the fact it increases death due to cancer. A third example is trials in cancer where interest lies in comparing treatments both in terms of their ability to prevent cancer recurrence and in terms of their adverse side effects, which may impact on the patient's quality of life. Any comparison of the treatments' effects on patient quality of life measures is complicated by the fact that inevitably such measures will be unavailable for some patients in each treatment group because they have died.

In the context of such issues, in recent years there has been increased scrutiny from drug regulatory agencies regarding how clinical trials specify how they will handle such complications in their design and statistical analysis. Specifically, there is an increased demand for trials to clearly specify exactly what kind of effect of treatment they seek to quantify (the so called estimand) and to choose a method of statistical analysis that handles these issues in a sensible and plausible manner.

The aim of this research is to investigate how such complications can best be handled using concepts and methods developed in the field of so called 'causal inference theory'. This theory offers a mathematical language to precisely describe what we mean by the effect of treatment in the presence of complicating factors such as the ones described earlier. Moreover, a large range of statistical methods have been developed for estimating treatment effects defined using these concepts, under different assumptions. This research will use causal inference theory to precisely define treatment effects (estimands) in the presence of the various issues described earlier. It will then investigate which statistical methods developed in causal inference theory are best suited for application to the analysis of clinical trial data.

The outputs of this research will help statisticians involved in clinical trials to use causal inference concepts and language to clearly specify the treatment effect which their trial intends to estimate. It will give them guidance and recommendations as to which statistical methods they can use to estimate such effects. The research will also produce software to implement the new statistical methods to enable trial statisticians to use the methods in their trials. Together these outputs will mean that patients can be offered more meaningful and accurate measures of expected treatment effects and that clinicians can make more informed decisions about patient care. The research will enable drug regulators and payer authorities to make fairer comparisons between treatments in regards their efficacy, safety, and cost-effectiveness, leading to improved decisions about which treatments to license and make available to patients.

In clinical trials a number of complications may arise which mean that a range of different types of treatment effect, or estimand, could be contemplated as the target of inference. For example, in trials where patients change treatments during follow-up (either as intended by the protocol or not), scientific interest may lie in estimation of the effects of a so called treatment regime which either was not adhered to in the trial or differs (e.g. for ethical reasons) from the trial's intended treatment regime. A second example is when the trial aims to assess effects on time to an event of interest but patients may experience a competing event first. A third example is where the trial aims to compare treatments' effects on some outcome but this outcome cannot be observed in a non-trivial proportion of patients because they have died (truncation by death). The recognition of the importance of clear specification of the estimand and appropriate choice of statistical method is evidenced by the commissioning of the ICH E9 addendum on clinical trial estimands.

The aim of this research is to apply the concepts and statistical estimation methods developed in modern causal inference theory to the problem of estimand definition and estimation in randomised trials. For a range of important estimand problems, we will use counterfactual language to precisely specify the estimand mathematically. We will use causal inference theory to characterize under what conditions such estimands can be identified or estimated from the observed data. Lastly we will critically evaluate, analytically, through simulation, and application to exemplar trial datasets, the different statistical methods which have been developed to estimate such estimands in regards the plausibility of their assumptions and their statistical properties. Open source software in R will be developed implementing these methods where implementations currently do not exist, with accompanying tutorials.

Statisticians working on the design and analysis of clinical trials will benefit from this research. This will include statisticians both within the pharmaceutical industry and statisticians working on publicly funded trials at universities, medical schools, and other institutions.

The peer reviewed journal publications resulting from this research will help trial statisticians use causal inference concepts and language to choose and more clearly specify the target estimand(s) in their trials. The ICH E9 addendum on estimands is expected to lead an increased emphasis on trial protocol documents clearly specifying the trial's estimand(s), and so the outputs from this research will influence the content of these sections of protocols.

The publications and tutorials resulting from the research will guide trial statisticians as to which statistical methods can be used to estimate their chosen estimands. The outputs should thus influence the choice of statistical analysis and how these are described in statistical analysis plans. The research should help make clear why a particular statistical method has been chosen given the trial's targeted estimand. This research project will produce R packages for methods deemed useful to trial statisticians where necessary, enabling them to implement these methods in practice and ensuring that the methods are actually used.

These impacts on the way trials are designed, documented, and analysed will lead to greater transparency and clarity regarding what the results of clinical trials mean for a range of stakeholders, including patients, clinicians, regulators, and payers. For patients and clinicians it should enable effect estimates to be provided that answer questions more relevant to them, such as 'what is the expected effect if I start and adhere to this treatment plan for the specified period?', rather than 'what is the expected effect of this treatment plan taking into account some patients will decide to withdraw from it?'. This will mean patients receive more accurate information about expected effects of treatments and this will help clinicians to make more informed decisions about patients' treatment and care. For regulators and payers the improvements should allow fairer comparison of effect estimates between different trials, through the anticipated clearer specification of trials' estimands and statistical analysis methods. This will enhance their ability to make sound evidence-based decisions about new treatments' efficacy, safety and cost effectiveness. It is anticipated that these impacts could be progressively achieved within five years of the research outputs being disseminated.