Therapist Effects in the Design, Analysis and Reporting of Psychotherapy Randomised Clinical Trials

Lead Research Organisation: King's College London
Department Name: Unlisted

Abstract

Clinical trials of psychotherapies make up a significant proportion of all clinical trials carried out in mental health. Yet relatively few explicitly take the therapist into account either in the design or in the analysis. This has serious implications for the precision, confounding and generalisability of the trial results and conclusions. The proposed research aims to (i) to review current understanding of therapist effects in the design, analysis and reporting of psychotherapy trials and systematic reviews of psychotherapy trials; (ii) to adapt and evaluate statistical approaches used in the context of cluster RCTs in order to incorporate therapist variation in meta-analyses of psychotherapy trials. The research will involve the secondary analysis of data from published trials and simulation work. It is expected that the new CONSORT guidelines for the reporting non-pharmacological trials will be evaluated in light of this work, as will existing MOOSE guidelines for the reporting of systematic reviews. This work is anticipated to help raise the quality of future clinical trials of psychotherapies and systematic reviews of psychotherapy trials.

Technical Summary

Background:

Patient outcomes in randomised trials that evaluate psychotherapies are clustered within therapists by design even in individually randomised trials. That is, outcomes for patients treated by the same therapist are expected to be more similar than outcomes for patients treated by different therapists. It is important that this is taken into account (i) in the design to ensure the trial is large enough to provide adequate statistical power to answer trial questions reliably and (ii) in the analysis to ensure the results and conclusions of such trials are valid. Generalisability of the results is also closely tied to the representativeness of the particular therapists providing treatment. Previous research has indicated that these issues have been infrequently acknowledged and even more rarely addressed adequately.

Aims & Objectives:

(i) To review design, analysis and reporting issues of particular relevance to psychotherapy trials in the context of explanatory versus pragmatic trial designs and paradigms for developing and evaluating psychotherapies.
(ii) To investigate the awareness and impact of therapist effects in the context of systematic reviews and associated meta-analyses of randomised trials of psychotherapies.
(iii) To investigate how trialists have addressed the internal and external validity issues arising from therapist effects and develop brief reporting guidelines to supplement the CONSORT statement.
(iv) To investigate the magnitude of therapist effects thereby allowing researchers to plan future trials more effectively; and
(v) To investigate how trialists can address problems in the analysis arising from non-compliance at the level of the therapist, changes in therapist effects over time, and the need for initial trials with small numbers of therapists.

Methods:

A combination of literature review, secondary analysis of previously published trial data and simulation methods will be used.

Scientific & Medical Opportunities:

A better understanding and awareness of the importance and context of the issues relating to therapist effects will enable guidelines to be developed (to supplement the CONSORT statement) across the full range of scenarios in which these issues arise. Explicit consideration of how they relate to paradigms for the development and evaluation of psychotherapies will clarify the different design characteristics of explanatory and pragmatic psychotherapy trials and enable guidelines to be developed to extend and further clarify the MRC‘s (2000) framework for the development and evaluation of RCTs of complex interventions to improve health. These guidelines will in turn motivate higher quality psychotherapy trials that are better able to answer important questions reliably.
 
Description MRC Skills Development Fellowship
Amount £235,559 (GBP)
Funding ID MR/N015444/1 
Organisation Medical Research Council (MRC) 
Sector Public
Country United Kingdom
Start 07/2016 
End 06/2019
 
Description Methodology Research Programme
Amount £251,787 (GBP)
Funding ID MR/P026761/1 
Organisation Medical Research Council (MRC) 
Sector Public
Country United Kingdom
Start 06/2018 
End 05/2020
 
Description NIHR Doctoral Research Fellowship - DRF-2018-11-ST2-079
Amount £299,709 (GBP)
Funding ID DRF-2018-11-ST2-079 
Organisation National Institute for Health Research 
Sector Public
Country United Kingdom
Start 09/2018 
End 08/2023
 
Description NIHR Pre-doctoral Fellowship
Amount £60,060 (GBP)
Funding ID NIHR300481 
Organisation National Institute for Health Research 
Sector Public
Country United Kingdom
Start 02/2020 
End 01/2022
 
Description NIHR Research Methods Fellowship
Amount £216,884 (GBP)
Funding ID RMFI-2013-04-004 
Organisation National Institute for Health Research 
Sector Public
Country United Kingdom
Start 09/2013 
End 08/2015
 
Description DoE 
Organisation King's College London
Country United Kingdom 
Sector Academic/University 
PI Contribution A collaboration has been formed with Professor Steven Gilmour regarding a programme of work on design of experiments for evaluating complex healthcare interventions - a day a month is spent on this currently
Collaborator Contribution Equal contribution is made by Professor Steven Gilmour
Impact Shortlisted for an EPSRC Centre for Mathematical Sciences in Healthcare in 2015 In 2021, awaiting outcome of NIHR Advanced Fellowship application and EPSRC Small Grant application
Start Year 2015
 
Description MH in Sheffield 
Organisation University of Sheffield
Department Centre for Psychological Services Research
Country United Kingdom 
Sector Academic/University 
PI Contribution Discussions around novel trial designs for evaluating psychological interventions
Collaborator Contribution Discussions around novel trial designs for evaluating psychological interventions
Impact Shortlisted for an EPSRC Centre for Mathematical Sciences in Healthcare in 2015
Start Year 2015
 
Description Partnership of Junior Health Analysts 
Organisation Health & Social Care Information Centre (HSCIC)
Country United Kingdom 
Sector Public 
PI Contribution I represent the University of Leeds on the steering group
Collaborator Contribution Together we foster collaborations and organise training events
Impact Our first event (15/11/2013) is on Data Linkage
Start Year 2013
 
Description Royal Statistical Society's Young Statisticians Section 
Organisation Royal Statistical Society
Country United Kingdom 
Sector Charity/Non Profit 
PI Contribution I am a founding member of the Section committee.
Collaborator Contribution I was part of a group organising training events for young statisticians
Impact I have helped to organise events aimed at career young statisticians.
Start Year 2008
 
Description Surgery Bristol 
Organisation University of Bristol
Country United Kingdom 
Sector Academic/University 
PI Contribution Discussions on novel trial designs for evaluating surgical interventions
Collaborator Contribution Discussions on novel trial designs for evaluating surgical interventions
Impact Shortlisted for EPSRC Centre for Mathematical Sciences in Healthcare
Start Year 2015