Exploratory trials of complex public health interventions: development of guidance for researchers and funders

Recognition is growing of the potential public health value of interventions to improve health behaviours such as diet, physical activity and substance use. Given the complex causality of these behaviours, improving public health often requires complex interventions which comprise multiple components interacting to produce change, with their actions constrained and facilitated by their interaction with the settings into which they are introduced. Given the cost of such interventions and the potential for unintended harms, drawing on evidence of what works and rigorously evaluating effectiveness is crucial.
Where feasible, randomised controlled trials (RCTs) represent the most robust means of examining effectiveness. However, complex interventions are often difficult to implement in a manner consistent with programme theory across multiple settings and target groups, whilst conducting and interpreting RCTs in real word settings is challenging. Hence, prior to committing to definitive evaluation, randomised or otherwise, attention is needed to establishing the feasibility and acceptability of intervention plans and intended evaluation designs.
The need to carefully examine these issues is increasingly recognised, with an increasing number of studies describing an objective of establishing the feasibility of full scale trials. Such studies are described under terms including 'exploratory trials', 'phase 2 trials' or 'piloting and feasibility' studies. However, there is currently a lack of consistent guidance for researchers preparing bids for such work, or for peer reviewers or funders, on the most important questions to address in such studies, key methodological considerations in addressing these questions, when it is more or less appropriate to use randomisation during this phase, and the criteria on which decisions should be made regarding whether and how to progress to a full scale definitive evaluation. Hence, as reflected by the range of terms under which such studies are described, their conduct and reporting is inconsistent, and the basis on which conclusions regarding whether or not to progress to a full-scale evaluation are made often unclear.
The proposed study will develop guidance for the conduct of exploratory trials through reviewing current practice and eliciting views of expert funders and researchers, beginning by using systematic review methodology to identify studies describing an intention to investigate the potential for a full scale trial (including phase 2 trials, exploratory trials or feasibility and pilot studies) and existing guidance on their conduct. We will also conduct a horizon-scanning exercise to identify recent methodological developments in the design of phase 2 trials and pilot and feasibility studies in other areas of clinical research, to ensure that the guidance learns from recent developments in these related areas.
A web-based Delphi process and consensus meeting will then be used to develop guidance on when exploratory trials should be conducted, what research questions they should address, key methodological considerations in the conduct of such studies and the range of criteria that should inform a decision to proceed to a full scale definitive trial. The guidance will then be actively disseminated to research funders, researchers and journal editors to maximise its impact on the commissioning, conduct and reporting of exploratory trials of complex public health interventions.

We shall use systematic reviews, a Delphi exercise and horizon scan to develop guidance on exploratory trials of complex public health interventions - those conducted before a full scale effectiveness trial to develop or refine the intervention, and to estimate key trial design parameters. We shall conduct 1) a scoping review of current practice, to identify the aims, terminology and range of methods used in exploratory trials; 2) a systematic review of current guidance and recommendations, to identify what is seen as good or bad practice, and the key gaps and areas of inconsistency. The reviews will use comprehensive database searches, supplemented by hand searches and citation tracking, double screening using pre-defined eligibility criteria, and narrative synthesis. We shall conduct a web-based Delphi exercise to identify expert consensus on key aspects of the purpose, design and conduct of exploratory trials, and the key areas of disagreement. Round 1 will produce a comprehensive list of issues, based on the reviews and experts' views; round 2 will validate this with a wider group of experts; round 3 will present a set of recommendations to gauge levels of agreement, and the basis for any disagreement. Each round will be followed by qualitative and quantitative analysis of responses to gauge levels of agreement and identify points of disagreement. After round 3 we shall draft guidance, and hold a consensus workshop to resolve outstanding disagreements and refine the document. We shall conduct a horizon scan of novel approaches to exploratory trials in medicine and surgery, to identify innovations that could be applied to complex public health interventions. We shall undertake scoping reviews, key informant interviews and two expert workshops. Outputs will include a database of exploratory trials of complex public health interventions, guidance on the design and conduct of such trials, and a research agenda for innovation in exploratory trials in public health.

Who will benefit from this research?
Given the methodological focus of the research, its direct impacts will primarily be felt amongst researchers attempting to conduct exploratory trials of complex interventions in population health, and amongst funding bodies and peer reviewers involved in the process of allocating resources to such studies and advising on the publication of their findings. However, it can be confidently anticipated that more distal beneficiaries of potentially substantial impact will be public health decision makers and commissioners, as well as a diversity of professionals and practitioners who implement policies and interventions that have an influence on health behaviours.

How will they benefit from this research?
Effective guidance will allow researchers to more efficiently and effectively develop and conduct exploratory trials, whilst providing peer reviewers and research funders with objective criteria against which to assess the quality of bids and publications. However, given that researchers and funders involved in such research are united by the common goal of finding effective strategies for improving population health, helping these stakeholders to improve their practice clearly has wider social and economic implications. Perhaps the most immediate benefit of assisting researchers and funders in efficiently focusing exploratory trials on the key issues to be addressed prior to definitive evaluation is that this may lower the public expense committed to the conduct of this specific research phase. However, such studies are not conducted as an end in themselves, but with the explicit purpose of informing decisions regarding whether and how to proceed to a definitive evaluation. Hence, perhaps the most important potential outcome of developing recommendations for this phase of the development and evaluation process is that it will likely play a significant role in increasing the chances of the most promising ideas worthy of future research being identified early and rigorous plans for their evaluation formed. In doing so, the risk of large volumes of public money being wasted on trials which are too poorly conducted for conclusions to be trusted, or on good quality trials of interventions which have little realistic prospect of achieving their goals, may be reduced.

Many of the most intractable problems of developed western societies relate to problems of human behaviour and the challenge to governments of effective and efficient means to change behaviour. Within health, non-communicable chronic disease and risk taking behaviour are critical issues and governments are urgently seeking to tackle problems such as obesity, diabetes, mental health, tobacco use, alcohol related disease and sequelae of binge drinking. It is increasingly understood from theoretical and empirical evidence that the most likely effective and sustainable solutions will involve complex multilevel interventions rather than more simplistic individual focussed interventions, and in order to identify, develop and evaluate the most effective of such interventions, and discard those that are intuitively appealing but nevertheless expensive, ineffective or harmful, improved research methods are required. This proposal will therefore in the medium term be likely to have substantial impacts on the health and wellbeing of the population and on the effectiveness and efficiency of preventive services and policies, by increasing rigorous and reliable evidence of what does and doesn't work.