HMT: Accuracy vs Precision - Developing optimal estimators for trials with multiple hypothesis tests

Accurate estimation of a treatment's effect is vital for expressing its true worth. This is important in early phase trials as this information is routinely used to decide whether further study is warranted and to power future studies. Additionally, health bodies require accurate estimation of the treatment effect to make informed choices about cost-effectiveness relative to alternative options as well as benefit risk assessment. In many current studies multiple hypothesis tests are conducted which has the unwanted side effect that the standard method for estimation is inaccurate.

This project aims to thoroughly evaluate the extent of bias of the standard estimator in studies in which multiple hypothesis tests are conducted. We will also investigate appropriate graphical displays to visualize treatment effects and develop novel approaches for point and interval estimation that balance accuracy and precision of the estimator. The developments will be made in close connection with clinical researchers and will be overseen by an advisory panel.

Accurate estimation of a treatment's effect is vital for expressing its true worth. This is important in early phase trials as this information is routinely used to decide whether further study is warranted and to power future studies. Additionally, health bodies require accurate estimation of the treatment effect to make informed choices about cost-effectiveness relative to alternative options as well as benefit risk assessment. The treatment effect is usually reported as the maximum likelihood estimator (MLE) which is a precise and readily available estimator. For designs that include interim analyses, it is, however, typically biased since it ignores the sequential nature of the trial. Additional bias is introduced by selecting treatments, subgroups or endpoints based on observed effect sizes.

To address this problem, several classes of estimation procedures have been proposed, most methods are, however, limited to two-stage designs and normally distributed observations. Additionally few estimators offer corresponding confidence intervals, limiting their usefulness in practice. This project aims to thoroughly evaluate the extent of bias of the MLE in studies in which multiple hypothesis tests are conducted and develop novel approaches for point and interval estimation. Specific objectives, which will be discussed in more detail below, are to:

- evaluate the extent of bias for a range of statistical designs;
- develop graphical approaches for displaying treatment effects for correlated hypotheses;
- develop point and interval estimates that yield an optimal trade-off of precision and accuracy;
- develop bias-reducing methods for estimating effects for time-to-event endpoints;
- develop open-source software and associated training tailored to clinical trialists and applied statisticians working on clinical trials.

To ensure practical relevance, the overall research programme will be overseen by an advisory board.

In addition to academic beneficiaries, the work proposed in this grant will have great impact on others. Evaluating the bias of the maximum likelihood estimator and balancing bias and variation in the estimation of treatment effects in studies with multiple hypothesis tests will be of great interest to those using results from a clinical trial. Those planning future trials will be able to use unbiased estimates when synthesizing evidence through techniques like meta-analysis and be able to power subsequent studies appropriately. Regulatory organisations such as the European Medicines Agency or the National Institute for Clinical Excellence will be able to judge the cost-effectiveness of a drug relative to alternative options as well as benefit risk assessment using reliable data.

The development of graphical tools for depicting treatment effects for subgroups and other correlated settings will benefit clinical trialists in communicating the findings to clinical experts by offering an easy to understand tool that does not require in-depth understanding of the underlying methods. Similarly these tools will allow medical doctors to communicate treatment decisions to patients that are member of subgroups that react differently to treatment more easily and effectively.