Design and Analysis of Trials of Group Treatments

Lead Research Organisation: University of Manchester
Department Name: Medical and Human Sciences

Abstract

Health service providers are under pressure to increase the availability of non-drug treatments for mental health and musclo-skeletal problems. Sometimes these treatments are delivered to patients in groups. Examples of such treatments include exercise classes for the treatment of back or joint pain, group therapies for mental health problems, and self-help groups for smoking cessation or alcohol problems.

It is important for patients and policy makers that the effectiveness and cost-effectiveness of this type of treatment are properly tested in well-designed randomised controlled trials, but there are reports of serious weakness in the design and statistical analysis of trials of this type of treatment.

Where a patient receives treatment in a group, the result of treatment may be more similar to a patient in the same group than to patients from other groups. This is because the group treatment may involve interaction between patients with some groups working well and some not. This variability in outcome between therapy groups is an example of clustering. To compensate for clustering, clinical trials of group treatments need to have larger numbers of groups. Unless special methods of statistical analysis are used to take account of clustering, the results of the trial may not be valid.

Trials of group treatments, such as these, are often compared with a treatment delivered to the patient individually. For example self-help groups for smoking cessation might be compared to an individualised counselling or use of nicotine patches. This creates special problems for statistical analysis, as standard methods assume clustering in both the treatment group and the control group. Another complications, is how should those patients who do not attend their therapy group be included in the statistical analysis of the trial.

The proposal will use computer simulation to test the performance of methods of statistical analysis. This involves repeatedly simulating the data, and then testing how well different methods of analysis work on the simulated data. Statistical methods will also be tested on real dataset from trials of group treatments.

When planning a trial of group treatments, information on the clustering effect is needed to calculate sample size. As part of this work we will carry out literature searches to find estimates of the clustering effect and dataset from which we can extract this information by reanalysing the data. These estimates will be made publicly available as a resource for researchers and trialists.

Technical Summary

The proposal is concerned with statistical issues in the design and analysis of trials of group administered treatments such as group therapies or self help classes.

The design and analysis of trials of group administered treatments should assume that between-therapy group variation in patient outcomes may occur. This has implications for design, including sample size estimation, and methods of statistical analysis, similar to the effects of clustering in cluster randomised trials.

Concerns have been raised regarding the methodological quality of such trials by several authors due to the inappropriate methods of statistical analysis or failure to take account of clustering due to therapy group in the sample size calculation. Often trials compare a group administered treatment with an individually delivered treatment. Examples of such trials include the comparison of group CBT with individual CBT for psychological disorders, the comparison of exercise classes with information giving, or the comparison of self help groups with individual treatment for smoking cessation. There are concerns regarding the performance of standard statistical methods that take account of clustering in this setting, particularly where numbers of therapy groups is small. There are also issues regarding the appropriate methods of statistical analysis to take account of clustering, where patients do not adhere to treatment. These issues and this trial design will be the main focus of this study.

This proposal will test and compare statistical methods for both intention-to-treat and causal analysis of trials of group versus individual therapy. The work will involve Monte Carlo simulation experiments and the analysis of data from trials of group therapies. This will include testing for bias in different methods for the estimation of the treatment and clustering effects, and also statistical test size. We will also empirically check the performance of method for sample size and power calculation.

Sample size calculation depends on estimates of the intra-cluster correlation coefficient (ICC) for therapy group, but few estimates are available in the published literature. The study will collate estimates of the ICC from reanalysis of datasets using the best methods identified. As part of this work we will carry out literature searches in order to identify estimates and gain access to dataset for reanalyses. These estimates will be useful to investigators planning trials of this type and will be made publicly available.

Publications

10 25 50
 
Description NIHR Methodology Fellowship
Amount £135,422 (GBP)
Organisation National Institute for Health Research 
Sector Public
Country United Kingdom
Start 09/2009 
End 12/2012
 
Description NIHR Research Methodology Fellowship
Amount £88,000 (GBP)
Organisation National Institute for Health Research 
Sector Public
Country United Kingdom
Start 05/2012 
End 05/2012
 
Title Cluspower Calculator / Clsampsi Calculator 
Description Software for estimating power and sample size for trials with clustering including group therapy trials. This has now been extended to directly calculate sample size and also binary outcome measure using a variety of methods 
Type Of Material Improvements to research infrastructure 
Year Produced 2009 
Provided To Others? Yes  
Impact We have been contacted by statisticians and health services who would like to us it. It is currently available at: http://www.medicine.manchester.ac.uk/healthmethodology/research/biostatistics/data/clsampsi 
URL http://www.medicine.manchester.ac.uk/healthmethodology/research/biostatistics/data/clsampsi
 
Title Cluspower Calculator / Clsampsi Calculator 
Description We have carried out improvements to the software to improve 
Type Of Material Improvements to research infrastructure 
Year Produced 2009 
Provided To Others? Yes  
Impact We have been contacted by statisticians and health services who would like to us it. It is currently available at: http://www.medicine.manchester.ac.uk/healthmethodology/research/biostatistics/data/clsampsi 
URL http://www.medicine.manchester.ac.uk/healthmethodology/research/biostatistics/data/clsampsi
 
Description Biostatistic Seminar 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Type Of Presentation Keynote/Invited Speaker
Geographic Reach Regional
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact 50 people attended this half day meeting of which with was one of three presentations related to developments in Multilevel modelling.

Questions and interest. Invitations to present the material elsewhere.
Year(s) Of Engagement Activity 2013
 
Description ISCB 2014 Vienna 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact Questions and discussions with other methodologists

Discussion of implication of research for the descign of cluster randomised trials
Year(s) Of Engagement Activity 2014
 
Description Invitated Presentation MRC Trials Methodology Conference Bristol 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? Yes
Type Of Presentation Keynote/Invited Speaker
Geographic Reach International
Primary Audience Health professionals
Results and Impact Invited Presentation to Conference

Discussions with statisticians and trials methodologust working on complex interventions.
Year(s) Of Engagement Activity 2011
 
Description RSS Cluster Randomised Trials Workshop 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact The was a workshop for members of the Royal statistical Society with an interest in cluster randomised trials
Year(s) Of Engagement Activity 2016
 
Description Seminar MRC Biostatistics Unit Cambridgde 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact This was a seminar to staff at the MRC Biostatistics Unit
Year(s) Of Engagement Activity 2016
 
Description Seminar: Royal Statistical Society 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Type Of Presentation Paper Presentation
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact Seminar to Lancashire Group of the Royal Statistical Society

Discussions regarding complex intervention trials.
Year(s) Of Engagement Activity 2011
 
Description Seminar: Royal Statistical Society 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact Presentation stimulated interesting discussion of implications of work

New ideas for possible collaborations
Year(s) Of Engagement Activity 2014
 
Description Seminar: University College of London 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact Seminar to Statisticians and Trials Methodologists at the University College of London

Stimulating discussion with methodologist unaware of the issue or its implications
Year(s) Of Engagement Activity 2014
 
Description Seminar: University of Leeds 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact Talk stimulated discussion and raised possibilities for collaboration.

Invited to give seminar to a wider group.
Year(s) Of Engagement Activity 2014
 
Description Seminar: University of Sheffield 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Type Of Presentation Paper Presentation
Geographic Reach Regional
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact Seminar to Statisticians and Trials Methodologists , University of Sheffield

Discuss with Collabortator and Trials Methodologists at SCHARR
Year(s) Of Engagement Activity 2011
 
Description Seminar: University of Warwick 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Type Of Presentation Paper Presentation
Geographic Reach Regional
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact Seminar at the University of Warwick

Collaboration with Methodologist at the Warwick Clinical Trials Unit
Year(s) Of Engagement Activity 2011
 
Description Workshop Presentation: Amsterdam Multilevel Workshop 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact International Workshop on Multilevel Modelling Amsterdam March 2011

Closer links with Dutch collegues working in similar subject areas
Year(s) Of Engagement Activity 2011