Therapist Effects in the Design, Analysis and Reporting of Psychotherapy Randomised Clinical Trials
Lead Research Organisation:
King's College London
Department Name: Institute of Psychiatry
Abstract
Clinical trials of psychotherapies make up a significant proportion of all clinical trials carried out in mental health. Yet relatively few explicitly take the therapist into account either in the design or in the analysis. This has serious implications for the precision, confounding and generalisability of the trial results and conclusions. The proposed research aims to (i) to review current understanding of therapist effects in the design, analysis and reporting of psychotherapy trials and systematic reviews of psychotherapy trials; (ii) to adapt and evaluate statistical approaches used in the context of cluster RCTs in order to incorporate therapist variation in meta-analyses of psychotherapy trials. The research will involve the secondary analysis of data from published trials and simulation work. It is expected that the new CONSORT guidelines for the reporting non-pharmacological trials will be evaluated in light of this work, as will existing MOOSE guidelines for the reporting of systematic reviews. This work is anticipated to help raise the quality of future clinical trials of psychotherapies and systematic reviews of psychotherapy trials.
Technical Summary
Background:
Patient outcomes in randomised trials that evaluate psychotherapies are clustered within therapists by design even in individually randomised trials. That is, outcomes for patients treated by the same therapist are expected to be more similar than outcomes for patients treated by different therapists. It is important that this is taken into account (i) in the design to ensure the trial is large enough to provide adequate statistical power to answer trial questions reliably and (ii) in the analysis to ensure the results and conclusions of such trials are valid. Generalisability of the results is also closely tied to the representativeness of the particular therapists providing treatment. Previous research has indicated that these issues have been infrequently acknowledged and even more rarely addressed adequately.
Aims & Objectives:
(i) To review design, analysis and reporting issues of particular relevance to psychotherapy trials in the context of explanatory versus pragmatic trial designs and paradigms for developing and evaluating psychotherapies.
(ii) To investigate the awareness and impact of therapist effects in the context of systematic reviews and associated meta-analyses of randomised trials of psychotherapies.
(iii) To investigate how trialists have addressed the internal and external validity issues arising from therapist effects and develop brief reporting guidelines to supplement the CONSORT statement.
(iv) To investigate the magnitude of therapist effects thereby allowing researchers to plan future trials more effectively; and
(v) To investigate how trialists can address problems in the analysis arising from non-compliance at the level of the therapist, changes in therapist effects over time, and the need for initial trials with small numbers of therapists.
Methods:
A combination of literature review, secondary analysis of previously published trial data and simulation methods will be used.
Scientific & Medical Opportunities:
A better understanding and awareness of the importance and context of the issues relating to therapist effects will enable guidelines to be developed (to supplement the CONSORT statement) across the full range of scenarios in which these issues arise. Explicit consideration of how they relate to paradigms for the development and evaluation of psychotherapies will clarify the different design characteristics of explanatory and pragmatic psychotherapy trials and enable guidelines to be developed to extend and further clarify the MRC‘s (2000) framework for the development and evaluation of RCTs of complex interventions to improve health. These guidelines will in turn motivate higher quality psychotherapy trials that are better able to answer important questions reliably.
Patient outcomes in randomised trials that evaluate psychotherapies are clustered within therapists by design even in individually randomised trials. That is, outcomes for patients treated by the same therapist are expected to be more similar than outcomes for patients treated by different therapists. It is important that this is taken into account (i) in the design to ensure the trial is large enough to provide adequate statistical power to answer trial questions reliably and (ii) in the analysis to ensure the results and conclusions of such trials are valid. Generalisability of the results is also closely tied to the representativeness of the particular therapists providing treatment. Previous research has indicated that these issues have been infrequently acknowledged and even more rarely addressed adequately.
Aims & Objectives:
(i) To review design, analysis and reporting issues of particular relevance to psychotherapy trials in the context of explanatory versus pragmatic trial designs and paradigms for developing and evaluating psychotherapies.
(ii) To investigate the awareness and impact of therapist effects in the context of systematic reviews and associated meta-analyses of randomised trials of psychotherapies.
(iii) To investigate how trialists have addressed the internal and external validity issues arising from therapist effects and develop brief reporting guidelines to supplement the CONSORT statement.
(iv) To investigate the magnitude of therapist effects thereby allowing researchers to plan future trials more effectively; and
(v) To investigate how trialists can address problems in the analysis arising from non-compliance at the level of the therapist, changes in therapist effects over time, and the need for initial trials with small numbers of therapists.
Methods:
A combination of literature review, secondary analysis of previously published trial data and simulation methods will be used.
Scientific & Medical Opportunities:
A better understanding and awareness of the importance and context of the issues relating to therapist effects will enable guidelines to be developed (to supplement the CONSORT statement) across the full range of scenarios in which these issues arise. Explicit consideration of how they relate to paradigms for the development and evaluation of psychotherapies will clarify the different design characteristics of explanatory and pragmatic psychotherapy trials and enable guidelines to be developed to extend and further clarify the MRC‘s (2000) framework for the development and evaluation of RCTs of complex interventions to improve health. These guidelines will in turn motivate higher quality psychotherapy trials that are better able to answer important questions reliably.
