Design and Analysis of Trials of Group Treatments
Lead Research Organisation:
University of Manchester
Department Name: Medical and Human Sciences
Abstract
Health service providers are under pressure to increase the availability of non-drug treatments for mental health and musclo-skeletal problems. Sometimes these treatments are delivered to patients in groups. Examples of such treatments include exercise classes for the treatment of back or joint pain, group therapies for mental health problems, and self-help groups for smoking cessation or alcohol problems.
It is important for patients and policy makers that the effectiveness and cost-effectiveness of this type of treatment are properly tested in well-designed randomised controlled trials, but there are reports of serious weakness in the design and statistical analysis of trials of this type of treatment.
Where a patient receives treatment in a group, the result of treatment may be more similar to a patient in the same group than to patients from other groups. This is because the group treatment may involve interaction between patients with some groups working well and some not. This variability in outcome between therapy groups is an example of clustering. To compensate for clustering, clinical trials of group treatments need to have larger numbers of groups. Unless special methods of statistical analysis are used to take account of clustering, the results of the trial may not be valid.
Trials of group treatments, such as these, are often compared with a treatment delivered to the patient individually. For example self-help groups for smoking cessation might be compared to an individualised counselling or use of nicotine patches. This creates special problems for statistical analysis, as standard methods assume clustering in both the treatment group and the control group. Another complications, is how should those patients who do not attend their therapy group be included in the statistical analysis of the trial.
The proposal will use computer simulation to test the performance of methods of statistical analysis. This involves repeatedly simulating the data, and then testing how well different methods of analysis work on the simulated data. Statistical methods will also be tested on real dataset from trials of group treatments.
When planning a trial of group treatments, information on the clustering effect is needed to calculate sample size. As part of this work we will carry out literature searches to find estimates of the clustering effect and dataset from which we can extract this information by reanalysing the data. These estimates will be made publicly available as a resource for researchers and trialists.
It is important for patients and policy makers that the effectiveness and cost-effectiveness of this type of treatment are properly tested in well-designed randomised controlled trials, but there are reports of serious weakness in the design and statistical analysis of trials of this type of treatment.
Where a patient receives treatment in a group, the result of treatment may be more similar to a patient in the same group than to patients from other groups. This is because the group treatment may involve interaction between patients with some groups working well and some not. This variability in outcome between therapy groups is an example of clustering. To compensate for clustering, clinical trials of group treatments need to have larger numbers of groups. Unless special methods of statistical analysis are used to take account of clustering, the results of the trial may not be valid.
Trials of group treatments, such as these, are often compared with a treatment delivered to the patient individually. For example self-help groups for smoking cessation might be compared to an individualised counselling or use of nicotine patches. This creates special problems for statistical analysis, as standard methods assume clustering in both the treatment group and the control group. Another complications, is how should those patients who do not attend their therapy group be included in the statistical analysis of the trial.
The proposal will use computer simulation to test the performance of methods of statistical analysis. This involves repeatedly simulating the data, and then testing how well different methods of analysis work on the simulated data. Statistical methods will also be tested on real dataset from trials of group treatments.
When planning a trial of group treatments, information on the clustering effect is needed to calculate sample size. As part of this work we will carry out literature searches to find estimates of the clustering effect and dataset from which we can extract this information by reanalysing the data. These estimates will be made publicly available as a resource for researchers and trialists.
Technical Summary
The proposal is concerned with statistical issues in the design and analysis of trials of group administered treatments such as group therapies or self help classes.
The design and analysis of trials of group administered treatments should assume that between-therapy group variation in patient outcomes may occur. This has implications for design, including sample size estimation, and methods of statistical analysis, similar to the effects of clustering in cluster randomised trials.
Concerns have been raised regarding the methodological quality of such trials by several authors due to the inappropriate methods of statistical analysis or failure to take account of clustering due to therapy group in the sample size calculation. Often trials compare a group administered treatment with an individually delivered treatment. Examples of such trials include the comparison of group CBT with individual CBT for psychological disorders, the comparison of exercise classes with information giving, or the comparison of self help groups with individual treatment for smoking cessation. There are concerns regarding the performance of standard statistical methods that take account of clustering in this setting, particularly where numbers of therapy groups is small. There are also issues regarding the appropriate methods of statistical analysis to take account of clustering, where patients do not adhere to treatment. These issues and this trial design will be the main focus of this study.
This proposal will test and compare statistical methods for both intention-to-treat and causal analysis of trials of group versus individual therapy. The work will involve Monte Carlo simulation experiments and the analysis of data from trials of group therapies. This will include testing for bias in different methods for the estimation of the treatment and clustering effects, and also statistical test size. We will also empirically check the performance of method for sample size and power calculation.
Sample size calculation depends on estimates of the intra-cluster correlation coefficient (ICC) for therapy group, but few estimates are available in the published literature. The study will collate estimates of the ICC from reanalysis of datasets using the best methods identified. As part of this work we will carry out literature searches in order to identify estimates and gain access to dataset for reanalyses. These estimates will be useful to investigators planning trials of this type and will be made publicly available.
The design and analysis of trials of group administered treatments should assume that between-therapy group variation in patient outcomes may occur. This has implications for design, including sample size estimation, and methods of statistical analysis, similar to the effects of clustering in cluster randomised trials.
Concerns have been raised regarding the methodological quality of such trials by several authors due to the inappropriate methods of statistical analysis or failure to take account of clustering due to therapy group in the sample size calculation. Often trials compare a group administered treatment with an individually delivered treatment. Examples of such trials include the comparison of group CBT with individual CBT for psychological disorders, the comparison of exercise classes with information giving, or the comparison of self help groups with individual treatment for smoking cessation. There are concerns regarding the performance of standard statistical methods that take account of clustering in this setting, particularly where numbers of therapy groups is small. There are also issues regarding the appropriate methods of statistical analysis to take account of clustering, where patients do not adhere to treatment. These issues and this trial design will be the main focus of this study.
This proposal will test and compare statistical methods for both intention-to-treat and causal analysis of trials of group versus individual therapy. The work will involve Monte Carlo simulation experiments and the analysis of data from trials of group therapies. This will include testing for bias in different methods for the estimation of the treatment and clustering effects, and also statistical test size. We will also empirically check the performance of method for sample size and power calculation.
Sample size calculation depends on estimates of the intra-cluster correlation coefficient (ICC) for therapy group, but few estimates are available in the published literature. The study will collate estimates of the ICC from reanalysis of datasets using the best methods identified. As part of this work we will carry out literature searches in order to identify estimates and gain access to dataset for reanalyses. These estimates will be useful to investigators planning trials of this type and will be made publicly available.