Exploratory trials of complex public health interventions: development of guidance for researchers and funders

Lead Research Organisation: University of Glasgow
Department Name: College of Medical, Veterinary &Life Sci

Abstract

Recognition is growing of the potential public health value of interventions to improve health behaviours such as diet, physical activity and substance use. Given the complex causality of these behaviours, improving public health often requires complex interventions which comprise multiple components interacting to produce change, with their actions constrained and facilitated by their interaction with the settings into which they are introduced. Given the cost of such interventions and the potential for unintended harms, drawing on evidence of what works and rigorously evaluating effectiveness is crucial.
Where feasible, randomised controlled trials (RCTs) represent the most robust means of examining effectiveness. However, complex interventions are often difficult to implement in a manner consistent with programme theory across multiple settings and target groups, whilst conducting and interpreting RCTs in real word settings is challenging. Hence, prior to committing to definitive evaluation, randomised or otherwise, attention is needed to establishing the feasibility and acceptability of intervention plans and intended evaluation designs.
The need to carefully examine these issues is increasingly recognised, with an increasing number of studies describing an objective of establishing the feasibility of full scale trials. Such studies are described under terms including 'exploratory trials', 'phase 2 trials' or 'piloting and feasibility' studies. However, there is currently a lack of consistent guidance for researchers preparing bids for such work, or for peer reviewers or funders, on the most important questions to address in such studies, key methodological considerations in addressing these questions, when it is more or less appropriate to use randomisation during this phase, and the criteria on which decisions should be made regarding whether and how to progress to a full scale definitive evaluation. Hence, as reflected by the range of terms under which such studies are described, their conduct and reporting is inconsistent, and the basis on which conclusions regarding whether or not to progress to a full-scale evaluation are made often unclear.
The proposed study will develop guidance for the conduct of exploratory trials through reviewing current practice and eliciting views of expert funders and researchers, beginning by using systematic review methodology to identify studies describing an intention to investigate the potential for a full scale trial (including phase 2 trials, exploratory trials or feasibility and pilot studies) and existing guidance on their conduct. We will also conduct a horizon-scanning exercise to identify recent methodological developments in the design of phase 2 trials and pilot and feasibility studies in other areas of clinical research, to ensure that the guidance learns from recent developments in these related areas.
A web-based Delphi process and consensus meeting will then be used to develop guidance on when exploratory trials should be conducted, what research questions they should address, key methodological considerations in the conduct of such studies and the range of criteria that should inform a decision to proceed to a full scale definitive trial. The guidance will then be actively disseminated to research funders, researchers and journal editors to maximise its impact on the commissioning, conduct and reporting of exploratory trials of complex public health interventions.

Technical Summary

We shall use systematic reviews, a Delphi exercise and horizon scan to develop guidance on exploratory trials of complex public health interventions - those conducted before a full scale effectiveness trial to develop or refine the intervention, and to estimate key trial design parameters. We shall conduct 1) a scoping review of current practice, to identify the aims, terminology and range of methods used in exploratory trials; 2) a systematic review of current guidance and recommendations, to identify what is seen as good or bad practice, and the key gaps and areas of inconsistency. The reviews will use comprehensive database searches, supplemented by hand searches and citation tracking, double screening using pre-defined eligibility criteria, and narrative synthesis. We shall conduct a web-based Delphi exercise to identify expert consensus on key aspects of the purpose, design and conduct of exploratory trials, and the key areas of disagreement. Round 1 will produce a comprehensive list of issues, based on the reviews and experts' views; round 2 will validate this with a wider group of experts; round 3 will present a set of recommendations to gauge levels of agreement, and the basis for any disagreement. Each round will be followed by qualitative and quantitative analysis of responses to gauge levels of agreement and identify points of disagreement. After round 3 we shall draft guidance, and hold a consensus workshop to resolve outstanding disagreements and refine the document. We shall conduct a horizon scan of novel approaches to exploratory trials in medicine and surgery, to identify innovations that could be applied to complex public health interventions. We shall undertake scoping reviews, key informant interviews and two expert workshops. Outputs will include a database of exploratory trials of complex public health interventions, guidance on the design and conduct of such trials, and a research agenda for innovation in exploratory trials in public health.

Planned Impact

Who will benefit from this research?
Given the methodological focus of the research, its direct impacts will primarily be felt amongst researchers attempting to conduct exploratory trials of complex interventions in population health, and amongst funding bodies and peer reviewers involved in the process of allocating resources to such studies and advising on the publication of their findings. However, it can be confidently anticipated that more distal beneficiaries of potentially substantial impact will be public health decision makers and commissioners, as well as a diversity of professionals and practitioners who implement policies and interventions that have an influence on health behaviours.

How will they benefit from this research?
Effective guidance will allow researchers to more efficiently and effectively develop and conduct exploratory trials, whilst providing peer reviewers and research funders with objective criteria against which to assess the quality of bids and publications. However, given that researchers and funders involved in such research are united by the common goal of finding effective strategies for improving population health, helping these stakeholders to improve their practice clearly has wider social and economic implications. Perhaps the most immediate benefit of assisting researchers and funders in efficiently focusing exploratory trials on the key issues to be addressed prior to definitive evaluation is that this may lower the public expense committed to the conduct of this specific research phase. However, such studies are not conducted as an end in themselves, but with the explicit purpose of informing decisions regarding whether and how to proceed to a definitive evaluation. Hence, perhaps the most important potential outcome of developing recommendations for this phase of the development and evaluation process is that it will likely play a significant role in increasing the chances of the most promising ideas worthy of future research being identified early and rigorous plans for their evaluation formed. In doing so, the risk of large volumes of public money being wasted on trials which are too poorly conducted for conclusions to be trusted, or on good quality trials of interventions which have little realistic prospect of achieving their goals, may be reduced.

Many of the most intractable problems of developed western societies relate to problems of human behaviour and the challenge to governments of effective and efficient means to change behaviour. Within health, non-communicable chronic disease and risk taking behaviour are critical issues and governments are urgently seeking to tackle problems such as obesity, diabetes, mental health, tobacco use, alcohol related disease and sequelae of binge drinking. It is increasingly understood from theoretical and empirical evidence that the most likely effective and sustainable solutions will involve complex multilevel interventions rather than more simplistic individual focussed interventions, and in order to identify, develop and evaluate the most effective of such interventions, and discard those that are intuitively appealing but nevertheless expensive, ineffective or harmful, improved research methods are required. This proposal will therefore in the medium term be likely to have substantial impacts on the health and wellbeing of the population and on the effectiveness and efficiency of preventive services and policies, by increasing rigorous and reliable evidence of what does and doesn't work.

Publications

10 25 50
 
Title Guidance for feasibility studies of complex public health interventions (the GUEST study) (Sharon Simpson) 
Description As part of the guidance development we collected Delphi data from a variety of stakeholders as well as data from expert interviews and an expert workshop. This information was used to develop the new MRC guidance on feasibility studies. 
Type Of Material Database/Collection of data 
Year Produced 2018 
Provided To Others? No  
Impact None as yet 
 
Title HELP trial dataset 
Description This dataset includes data from pregnant women and women in the postpartum period. Data were collected at baseline and at 36 weeks gestation and at 6 weeks, 6 months and 12 months postpartum. Body Mass Index at 12 months postpartum was the primary outcome. Secondary outcomes include pregnancy weight gain, quality of life, mental health, waist-hip ratio, child weight centile, admission to neonatal unit, diet, physical activity levels, pregnancy and birth complications, social support, self-regulation and self-efficacy. Data on cost effectiveness and process evaluation data were also collected. 
Type Of Material Database/Collection of data 
Year Produced 2014 
Provided To Others? No  
Impact None as yet 
 
Description Personalised Exercise-Rehabilitation for people with multiple long term conditions (multimorbidity) The PERFORM trial 
Organisation University of Birmingham
Country United Kingdom 
Sector Academic/University 
PI Contribution We have collaborated on putting together this application for a NIHR Programme Grant. We are leading on the process evaluation and feasibility trial work packages.
Collaborator Contribution Our partners in Leicester are leading the grant, other partners are leading different workpackages.
Impact Funding application is about to be submitted.
Start Year 2019
 
Description Personalised Exercise-Rehabilitation for people with multiple long term conditions (multimorbidity) The PERFORM trial 
Organisation University of Exeter
Country United Kingdom 
Sector Academic/University 
PI Contribution We have collaborated on putting together this application for a NIHR Programme Grant. We are leading on the process evaluation and feasibility trial work packages.
Collaborator Contribution Our partners in Leicester are leading the grant, other partners are leading different workpackages.
Impact Funding application is about to be submitted.
Start Year 2019
 
Description Personalised Exercise-Rehabilitation for people with multiple long term conditions (multimorbidity) The PERFORM trial 
Organisation University of Leicester
Department Leicester Clinical Trials Unit
Country United Kingdom 
Sector Academic/University 
PI Contribution We have collaborated on putting together this application for a NIHR Programme Grant. We are leading on the process evaluation and feasibility trial work packages.
Collaborator Contribution Our partners in Leicester are leading the grant, other partners are leading different workpackages.
Impact Funding application is about to be submitted.
Start Year 2019
 
Description Personalised Exercise-Rehabilitation for people with multiple long term conditions (multimorbidity) The PERFORM trial 
Organisation University of Leicester
Department Leicester Medical School
Country United Kingdom 
Sector Academic/University 
PI Contribution We have collaborated on putting together this application for a NIHR Programme Grant. We are leading on the process evaluation and feasibility trial work packages.
Collaborator Contribution Our partners in Leicester are leading the grant, other partners are leading different workpackages.
Impact Funding application is about to be submitted.
Start Year 2019
 
Description Personalised Exercise-Rehabilitation for people with multiple long term conditions (multimorbidity) The PERFORM trial 
Organisation University of Salford
Country United Kingdom 
Sector Academic/University 
PI Contribution We have collaborated on putting together this application for a NIHR Programme Grant. We are leading on the process evaluation and feasibility trial work packages.
Collaborator Contribution Our partners in Leicester are leading the grant, other partners are leading different workpackages.
Impact Funding application is about to be submitted.
Start Year 2019
 
Description Personalised Exercise-Rehabilitation for people with multiple long term conditions (multimorbidity) The PERFORM trial 
Organisation University of York
Country United Kingdom 
Sector Academic/University 
PI Contribution We have collaborated on putting together this application for a NIHR Programme Grant. We are leading on the process evaluation and feasibility trial work packages.
Collaborator Contribution Our partners in Leicester are leading the grant, other partners are leading different workpackages.
Impact Funding application is about to be submitted.
Start Year 2019
 
Description Physical activity of young children 
Organisation University of Strathclyde
Country United Kingdom 
Sector Academic/University 
PI Contribution We collaborated on submitting two funding applications one to Wellcoma and the other to CSO.
Collaborator Contribution They led the funding application bids.
Impact None as yet.
Start Year 2019
 
Description Delivered a talk and expert panel discussion on developing and evaluating complex interventions and the update to the MRC guidance (Sharon Simpson) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Policymakers/politicians
Results and Impact Delivered a talk and expert panel discussion on developing and evaluating complex interventions and the update to the MRC guidance. The audience incldued a mix of academic and policy makers and practitioners.
Year(s) Of Engagement Activity 2019
 
Description Festival of Social Science (Susan Browne) 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact Festival of Social Science 2017 Ikea event. Best Start in Life activity. Intended to promote discussion and awareness of early years research taking place at SPHSU.
Year(s) Of Engagement Activity 2017