Improving the statistical efficiency of randomised controlled trials through their design

Randomised controlled trials (RCTs) are a way of assessing whether medical treatments are effective or not, in which patients are randomly allocated to receive the treatment or to a comparison group and their outcomes compared. RCTs are considered to be the gold-standard method of comparing treatments, but they are expensive and time consuming to conduct. Designing them well is therefore crucial. One way of improving the design of RCTs is to increase their statistical efficiency, which is a measure of the ability of an RCT to detect a treatment effect if one is present. If we can design more statistically efficient RCTs, then it would be possible to run them with fewer patients or for a shorter length of time, which will also reduce costs. The major objectives of this fellowship are to extend existing methods to compare the statistical efficiency of different RCT designs, and to explore the effect of different designs in several case-studies.

One way of increasing the statistical efficiency of an RCT is to measure a patient's outcome more than once. When planning such an RCT, it is necessary to make choices about design features such as the number and spacing of follow-up visits. The first aim of this fellowship is to investigate, and extend as necessary, existing methods for assessing the efficiency of different RCT design features. The methods will be applied to several case studies, including data from two RCTs - one of multiple sclerosis patients and the other looking at the effect of statins on muscle pain.

Many RCTs suffer from missing data, for example if some patients miss a follow-up visit. In addition, patients do not always receive the treatment they are assigned, for example if they forget or do not want to take their treatment. The second aim of this project will be to extend the aforementioned methods to cope with these situations, and to investigate how the optimal choice of design features changes in these circumstances.

The final aim of the fellowship is to explore what types of data can be used to plan the design of future RCTs. There may not be any data from previous RCTs available to help design future ones, but there might be routinely collected data sources, such as disease registries and electronic health records. However, it is not clear whether such data sets can be used to help choose design features. This fellowship will use two case-studies to assess the feasibility of using routinely collected data to design RCTs. The first case-study will use two different sources of data from patients with Huntington's disease: one which is collected in a very controlled manner, and the other which is a disease registry. The second case-study will use data from primary care electronic health records.

By developing methods to assess the statistical efficiency of different design features, this research will provide tools and information that can be used by researchers who design RCTs in the future.

Randomised controlled trials (RCTs) are an important tool for assessing the effectiveness of medical interventions. RCTs are expensive and time consuming, and designing more statistically efficient trials is therefore crucial. The primary objectives of this research are to develop and extend existing methodologies to assess the statistical efficiency of RCT designs, and compare the efficiency of different design features. This will be achieved through addressing the following questions:

1. How do design features, such as the number of interim visits, length of follow-up, and use of complex designs such as run-in periods, affect the statistical efficiency of an RCT?
2. What effect does missing data and/or non-compliance have on the optimal choice of such design features?
3. Is it possible to use routinely collected data such as electronic health records and disease registries to inform the optimal design of RCTs?

These questions will be addressed by developing and extending methods for assessing the statistical efficiency of RCTs, including existing methodology based on linear mixed models. The methods will be explored using both simulation and application to case-studies, including RCT data from multiple sclerosis patients and a series of N-of-1 trials, and observational data from patients with Huntington's disease. This research will focus on two types of RCT: those which have an outcome that is a rate, and series of N-of-1 trials. These types of design are of particular interest since they can offer greatly improved statistical efficiency compared with standard alternatives. Due to the complexity of these designs, there are many choices to be made about the design features. Developing methods that allow the optimal design features to be chosen offers the opportunity to maximise the efficiency of these designs.

The proposed research will provide information and methodologies that can be used to design statistically efficient RCTs.

Randomised controlled trials (RCTs) are the gold-standard method for assessing whether medical interventions are effective or not, but they are expensive to conduct. Recruitment to RCTs can be difficult and many trials fail to meet their recruitment targets, resulting in under-powered trials with too few patients. One consequence of lack of power is that RCTs investigating truly effective treatments may fail to give statistically significant results; these treatments may then not be used in practice. Another is that under-powered RCTs which find statistically significant benefits are likely to over-estimate treatment effect sizes, as explained by Button et al. in "Power failure: why small sample size undermines the reliability of neuroscience", Nature Rev Neuroscience 14 (2013) p365. This overestimation could lead to treatments being adopted in clinical practice which in fact are not as effective as they are thought to be, and may not meet cost-effectiveness requirements set by bodies such as NICE. Given these points, it is crucial to design RCTs with statistical efficiency in mind, and to improve efficiency through careful choice of design features. The former will ensure RCTs have appropriate power, and the latter will enable them to be run with fewer patients or for a shorter time.

This fellowship therefore has the potential to have a large impact, both within academic circles and more widely. Patients would be major beneficiaries of this research. Designing statistically efficient trials can lead to shorter RCTs, in which case information about the most effective treatments could be delivered to patients and the clinicians who treat them more quickly. Patients would also benefit through RCTs being run with a smaller sample size, since fewer patients would be exposed to potentially harmful interventions.

Other beneficiaries of the project would include researchers involved in planning and running RCTs, as well as statisticians with an interest in RCT design and methodology. This research would provide tools and information which would enable trialists to run more statistically efficient RCTs. Potential beneficiaries would therefore also include research councils, charities and other bodies who fund RCTs. If more statistically efficient trials are designed, this will enable RCTs to be smaller or shorter. This would reduce costs, and RCT funders would get more results for their money.

An additional benefit to both patients and researchers would be having fewer underpowered trials, either through insufficient sample sizes being used or due to recruitment targets not being met. As explained above, this can lead to true treatment effects being either missed, or overestimated if statistically significant. If RCTs are designed with appropriate statistical power this is less likely to happen.

The research relating to N-of-1 trials would be of particular benefit to the management of chronic diseases. The primary aim of conducting N-of-1 trials is to communicate to patients which treatment is best for them - a step towards personalised medicine. Designing statistically efficient N-of-1 trials would provide direct benefits to patients and the clinicians who treat them.

Results obtained by applying the methods to case studies would have an impact within the specific disease area of those studies. For example, when applying the methods to data from MS-STAT, an RCT in patients with secondary progressive multiple sclerosis (MS), valuable information as to which design features are statistically efficient would be gained. This information would be of great value when designing other MS trials. The results may have limited generalisability to other disease areas: however, the methods that would be developed and extended as part of this fellowship would have a wider generalisability and impact. Researchers could take these methods and apply them to other disease areas, and use them to design RCTs in other settings.