Evaluation of Bayesian adaptive designs for Phase 3 effectiveness trials

Clinical trials are time consuming, difficult and expensive, and represent a substantial barrier to improving treatments for patients. Researchers and funders have recognised the need for trials to become more efficient, yet the overwhelming majority of trials continue to use traditional methods. A different approach to trials, Bayesian adaptive trial methodology, has been developed over the last 20 years, and has the potential to allow trials to answer their questions more efficiently, often meaning that effectiveness can be determined with fewer patients and in a shorter time. This approach uses Bayesian statistical methods, and includes frequent interim analyses, to allow learning from the trial data collected so far, and subsequent modification of the trial. For example, the proportion of participants allocated to each intervention could be changed to ensure that participants are allocated most efficiently, randomisation to one arm could be stopped, if it has become clear that it is not effective, or the whole trial could be terminated early if it has achieved its aim. These adaptations may enable trials to be run more efficiently, and allow evaluation of more treatments with the same resources.
Our aim in this project is to evaluate Bayesian adaptive trial methods, to find out whether they are likely to lead to practical benefits for triallists and funders, in terms of their size, duration and cost. If so, there would be a strong case for using this methodology more widely, and it could potentially become the standard methodology in the future.
To evaluate these methods we will perform three studies using data from trials run by the Warwick Clinical Trials Unit. In Study 1, each trial will be redesigned using Bayesian adaptive methods, which will involve producing a number of candidate designs, then performing extensive simulations to understand their performance and to fine-tune their operational parameters such as number and timing of analyses, and threshold values for decisions. A single preferred design will be selected by a group of clinicians and researchers who were not involved in the original trial. The trial will then be re-run, using the original sequence of patients, and we will compare the re-run trial with the original overall duration, number of patients, results, number of patients receiving the more favourable treatment, and cost of the trial. Study 1 will involve four case studies, using trials that had different issues: one that was stopped early because the intervention appeared harmful, one multi-armed trial, one that concluded superiority of its intervention, and one that recruited for a long time but showed no difference.
Study 2 will use the same methods as Study 1, but will involve trials that have completed recruitment but have not yet been reported; their results will therefore not yet be known.
Study 3 will involve re-designing trials that use a conventional design and are starting or in the early stages of recruitment. In parallel with the real-life trial conduct, we will performing Bayesian interim analyses according to the redesigned schedule as the trial proceeds, in real time. This will show whether the Bayesian adaptive design would lead to different conduct, results and conclusions from the conventional design. No results from these "shadow" analyses will be released until the trial has concluded.

In this project we aim to evaluate Bayesian adaptive trial methodology, by re-designing and re-executing trials funded by the NIHR HTA Programme and conducted by Warwick Clinical Trials Unit (WCTU). Al of the datasets are held in anonymised format by WCTU. We plan three studies, using 1. datasets from completed trials; 2. datasets from trials that have completed recruitment but have not reported their results; 3. ongoing trials.
In studies 1 and 2, we will redesign each trial using the following potential adaptive features: 1. Interim analyses and potential early termination of the trial; 2. Response adaptive randomisation; 3. Dropping of arms; 4. Bayesian final analysis. For each trial we will perform extensive simulations to understand the performance of proposed designs and to optimise parameters such as the number of interim analyses and thresholds for decisions. A group of researchers and clinicians, independent of the original trial, will discuss the design with the project researchers and select a single design will be selected. The trial will then be re-run using the original patient sequence, to evaluate whether use of the Bayesian adaptive design would lead to practical benefits in terms of the size, duration or cost of the trial, or differences in results or conclusions. We will use resampling from the trial dataset to estimate the range of possible sample sizes and results under the alternative design.

In Study 3, we will carry out the same trial redesign process, but we will use data from real clinical trials in real time. The interim analyses will be carried out on the real accumulating data but the "decisions" resulting from them will not be implemented in the real trials. Results from the re-executions of these trials will be kept confidential and will not be made public until the real trial results are known.

We believe that this project could have substantial impact on the conduct of clinical trials in the UK and around the world. There is an urgent need to improve the efficiency of trials so that more treatments can be evaluated with the same resources. Virtually all trial proposals submitted to NIHR HTA and EME funding schemes are for traditional two-armed non-adaptive trials using standard frequentist methods, and innovative designs such as Bayesian adaptive methods are rarely considered. One of the reasons for this is that they are considered unproved and risky. If we demonstrate that there may be advantages to using this methodology, this is likely to encourage more use of these methods in the future. Also, at present there is a lack of skills in Bayesian adaptive trial methodology. We anticipate that through this project we will publicise the methods and demonstrate that they can be applied relatively easily.
The other main target audience for this research is funders of clinical trials, for whom a major concern is efficiency and maximising the return in terms of patient benefit from their limited resources. If the project demonstrates that Bayesian adaptive trials are likely to lead to gains in efficiency, we would expect funding bodies to become keen to fund this type of trial. A key element in their ability to fund innovative research designs is the willingness of funding boards to support them. At present innovative designs are seen as risky and are less likely to be supported. A key part of our strategy is to disseminate the results to funders, and offer to make presentations to funding boards and funders' administrative teams. By education, we hope to reassure funders that these designs are scientifically sound and should not be seen as risky. In fact, because of the extensive simulation involved in designing a Bayesian adaptive trial, their performance is much better understood than is the case for standard trial designs.
The ultimate beneficiary from this research is patients, who are the end-users of health research. By evaluating these designs we hope to be able to determine whether they will benefit patients in two ways: first, by improving research studies so that more patients receive favourable interventions during their evaluation, and trials reach sound conclusions with fewer adverse outcomes. Second, by increasing the number of trials that can be conducted, the overall evidence base for clinical practice will be enhanced.