Development of a fully Bayesian framework for the identification and estimation of subgroup effects in Randomised Controlled Trials

Population health can be improved by identifying treatments that work particularly well for individual patients. This is commonly described as stratified, precision, or personalised medicine. Personalised medicine is underpinned by subgroup analyses where the effectiveness of treatments are assess in subsets of study patients with defined characteristics. However, we need to account for the effects of chance, if we compare treatment response in a large number of subgroups we are likely to observe differences that arise purely by chance. For example, a study reported in 2005 listed ten cases where apparently clinically important subgroup effects were subsequently been shown to be false, such as the claim that aspirin for secondary prevention of stroke was ineffective in women. Unreliable subgroup analysis can lead to patients suffering harm, the waste of scarce resources, and futile confirmatory trials.
The conventional approach is to advise caution, apply qualitative checklists to assess credibility, and apply statistical corrections. However, these approaches do not directly address the biases caused by multiplicity and leave the burden on the decision-maker to balance the risk of accepting "spurious" subgroup effects against the risk of rejecting true subgroup effects. This challenge can be seen in the FDA deliberations over the PLATO trial, where a treatment effect appeared to reverse in North American patients, and the differing decisions made by NICE and IQ-WiG based on subgroup analysis from the CAPRIE trial. Prior plausibility is identified as an important criterion for the credibility subgroup effects, but there is little guidance regarding how and when these judgements should be elicited. They are rarely elicited and recorded before trials commence, leaving decision-makers attempting to judge plausibility after trial results have been published.

In this project, we will develop a fully Bayesian framework for subgroup analysis comprising a statistical model that adjusts for the effects of multiple testing and a framework for elicitation of judgements regarding the plausibility of subgroup effects. Prior plausibility is an important criterion for judging the credibility of subgroup effects but there is little guidance as to how and when plausibility should be assessed and reported.
Importantly the statistical model will allow for the observed subgroup effects to represent a mixture of "true" reproducible subgroups and spurious effects caused purely by random chance. The Bayesian model will provide a direct estimate of the probability that a subgroup effect occurred by chance and will also be able to incorporate informative priors representing belief in the plausibility of subgroup effects.

The framework will be tested in a number of real life case studies representing study design, commissioning of confirmatory studies based on subgroup analysis, and technology appraisal. Guidance on the use of the framework, to facilitate translation, will be developed based on the case studies.

We will develop a fully Bayesian framework for subgroup analysis encompassing both a novel statistical modelling approach (objective 1) and a systematic web-based framework for the elicitation of expert opinion on subgroup effects (objective 2). The framework will be refined using a number of case studies. To facilitate translation, guidance will be developed based on the case studies describing the use of the framework (objective 3). The statistical model is central to our approach. Our key proposal is not simply to apply shrinkage, but to explicitly incorporate a mixture component to differentiate reproducible subgroup effects from chance effects. This is similar to the use of slab and spike priors for variable selection. The hierarchical Bayesian framework allows direct inclusion of information elicited from experts. It also naturally extends to allow information on subgroup effects to be "shared" across studies and endpoints and can incorporate either individual patient data or reported summary statistics. To understand the performance of our models in real live settings, we will compare the (i) sensitivity and specificity of the identification of subgroup effects and (ii) bias and interval coverage for estimation of identified subgroup effects in a series of simulation studies. We will develop web based tools that will facilitate clearer, quantifiable and readily interpretable statements of the prior plausibility of subgroup effects. These tools, and methods of elicitation, will be tested during our case studies. In particular we will test various scales for elicitation to determine which are meaningful to experts. In order to support the practical use of the tools developed under objectives 1 and 2, we will develop guidance describing and critically examining the use of the statistical model and elicitation tools in three distinct stages of technology development: (i) clinical trial planning (ii) the consideration of further studies based on subgroup analysis

The proposes Bayesian framework for subgroup analysis will benefit those designing and conducting trials, those making treatment and funding decisions, and ultimately patients. The framework will help ensure that the maximum value is obtained from the investment in trials by increasing the reliability and impact of any subsequent subgroup analysis. This will increase the return on investment on in clinical research both in the public and private commercial settings. More reliable and impactful subgroup analysis will help public sector funders and regulators make better use of scarce health resources by targeting therapy at patients who will gain the most benefit. It will also help funders identify and recommend the most cost-effective treatments for specific groups of patients. Specifically it will aid regulatory and other decision-makers in the difficult task of balance the risk of accepting "spurious" subgroup effects against the risk of rejecting true subgroup effects. More reliable targeting of therapies, particularly early in the development process, will help technology developers more accurately assess the potential future value of therapies and hence make better investment decisions. This will potentially benefit both the private and public sector. We will specifically consider the use of subgroup analysis when commissioning of further studies. Unreliable subgroup analysis at this point can lead to the significant waste of resource on futile confirmatory trials. Decision making at this juncture is particularly vulnerable to cognitive decision biases such as optimism, anchoring and availability bias. The quantitative Bayesian framework will help to address these biases.
Ultimately the research can contribute to the nation's health by improving both treatment and investment decisions by enabling more reliable targeting of treatments. By improving the efficiency of technology development, the research may improve the economic competitiveness of the United Kingdom's healthcare industry. The proposed Bayesian framework will be suitable for implementation during the funded period and dissemination activities towards the end of the project will be focussed on ensuring translation into actual use.