Adaptation of evidence-informed complex population health interventions for implementation and/or re-evaluation in new contexts: New guidance

Lead Research Organisation: Cardiff University
Department Name: Sch of Social Sciences

Abstract

Complex population-health interventions that are effective in one context may not be effective in a different context, and may even be harmful. This is largely because intervention components work differently across different contexts. As such, an intervention may require adaptation to ensure it fits with a new setting. Further evaluation may be necessary to establish if intervention effects can be replicated. To date there is no guidance to support researchers to adapt and evaluate interventions in new contexts. There is no criteria to support research funders or peer reviewers in deciding upon the appropriateness of proposed or reported adaptations or evaluation. Further, there is limited instruction for policy-makers and practitioners to decide if evidence-informed interventions are appropriate to their context, or if adaptation and further evaluation is needed. This study will provide guidance to these communities to support the adaptation, implementation and/or re-evaluation of complex population health interventions in new contexts. It is important to provide guidance to ensure that interventions are adapted and evaluated using the most appropriate methods. This would minimise public spending on interventions that are not likely to be effective in new contexts due to unsuitable adaptations. It will also reduce the funding of large scale evaluations of interventions in new contexts, where uncertainty about their likelihood of working is insufficient to warrant their expense.

We will review existing guidance and examples where complex population-health interventions have been adapted to new contexts. This will help us to understand current practice and to explore how adaptations are made and justified. It will also improve understanding of how guidance is used and how intervention successes or failures are explained in relation to its fit with the new context. We will also conduct interviews with researchers, representatives of international level research funders, peer reviewers, policy makers and practitioners. Participants will be identified from guidance and studies retrieved during the reviews. We will also involve contacts identified by the study's research team. Selected participants will have experience of commissioning, delivering or evaluating evidence-informed population health interventions that have been adapted or implemented in a new context. Interviews with researchers, policy-makers and practitioners will explore how decisions are made and justified around intervention adaptation, the procedures followed, and the use of guidance. Interviews with research funders and peer reviewers will focus on the appropriateness of proposed and reported adaptations and evaluation. We will subsequently complete a web-based Delphi process and consensus meeting to develop adaptation guidance. This will involve three rounds of surveys with experts to decide which recommendations should be included in the guidance and where further research may be required as there is insufficient consensus. Following the surveys we will draft the guidance. A meeting will then be held with experts to reach agreement on, and finalise, the guidance. Once completed, the guidance will be disseminated to researchers, research funders, peer reviewers, policy-makers and practitioners to maximise impact on the way in which intervention adaptations are funded, undertaken and reported.

Technical Summary

Complex population health interventions that demonstrate effectiveness in one context may be ineffective or harmful elsewhere. Intervention adaptation and re-evaluation may be required. To date, there is no established guidance for the adaptation, implementation and/or re-evaluation of complex interventions in new contexts. This study will develop guidance for researchers, research funders, journal editors, policy-makers and practitioners. The study will comprise four work packages. WP1 will include systematic review of current guidance. It will proceed to a mapping review of studies reporting intervention adaptation, implementation and/or revaluation in new contexts. This will explore how researchers undertake and explain adaptations, and how they utilise guidance. WP2 will include qualitative interviews with researchers, representatives of international-level funding bodies, peer reviewers, policy makers and practitioners. Interviews will build on WP1 and examine decision-making around intervention adaptation, the procedures undertaken, and the uptake of guidance. Interviews with research funders and peer reviewers will focus on the perceived appropriateness of proposed or reported adaptations and evaluation. WP3 will comprise a web-based Delphi exercise to identify expert consensus on adaptation. Round 1 of the survey will generate a comprehensive list of items to be considered; round 2 will validate these items with a wider group of expertise; and round 3 will present recommendations for guidance inclusion. Following the survey, guidance will then be drafted and we will host a consensus meeting to gauge agreement and identify areas that may require further development. WP4 will oversee study coordination and dissemination. Study outputs will include guidance on the adaptation, implementation and/or re-evaluation of population health interventions in new contexts, and a research agenda for intervention adaptation in population health.

Planned Impact

The study will have significant impact across a range of beneficiaries. Further details of study beneficiaries, and the intended mechanisms for achieving this impact, are presented in the 'Pathway to Impact' statement. The applicants have an excellent track record of ensuring the uptake of guidance; process evaluation guidance led by Moore (PI) has been cited almost 500 times since 2015, while MRC guidance led by Craig (2008) has been cited almost 4000 times.

1. Who will benefit from the research?
The primary focus of the study is to improve methodological conduct in the adaptation, implementation and/or re-evaluation of interventions in new contexts. As such, the most immediate beneficiaries will be population health researchers engaged in intervention adaptation, and funding bodies and journal editors with a remit for commissioning adaptation research and disseminating research findings. The study will further impact upon policy-makers and practitioners who commission interventions within their context or have responsibility for delivering population health approaches and may be considering adaptation. More distal beneficiaries will include the general population, who may experience improved health and wellbeing.

2. How will they benefit from the research?
Development of guidance will enhance researchers' knowledge and expertise in the adaptation, implementation and/or re-evaluation of complex population interventions in new contexts, thus promoting the conduct of scientifically robust research that utilises the most appropriate processes and methodologies. It will provide research funders and journal editors with criteria against which to assess the appropriateness of proposed or undertaken adaptations, as well as the evaluation methodologies. This will ensure the resourcing and dissemination of the highest quality research. Beyond the research community, the study will impact policy-makers and practitioners, who are key participating experts in the Delphi consensus exercise. A bespoke version of guidance for policy and practice communities will support them in assessing the need for adapting complex population health interventions as they are commissioned and implemented within their local context, and to inform the need to conduct further evaluation. Guidance will also support development of research capacity within these communities.

Improvement in how research, policy and practice communities commission, conduct and disseminate intervention adaptations will likely lead to more effective interventions that directly impact upon health and wellbeing at the population level. The study will contribute to the minimisation of resource expenditure on interventions that will likely not replicate effectiveness in new contexts as they have not been appropriately adapted. As such, public finance can be dedicated to interventions that have a realistic prospect of demonstrating impact. Equally, the study will allow for ascertainment of the adequate level of re-evaluation for interventions in new contexts to ensure sufficient and robust evidence of effectiveness is generated. This may prevent replication of effectiveness evaluations (e.g. RCTs), which require significant financial resource, when contexts are sufficiently similar that evidence from the original trial may provide adequate certainty of effects in the new setting. Therefore, the medium to long-term impacts of the study will be the improvement of population health and wellbeing through the development of more contextually appropriate interventions, with this being achieved alongside the more efficient allocation of public resource.
 
Description Contribution to international short course teaching 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact Contribution to short course teaching in Waginengen based around plans for MRC-NIHR funded study on adaptation of interventions across contexts

https://www.vlaggraduateschool.nl/en/courses/Scientific-courses.htm
Year(s) Of Engagement Activity 2018
 
Description European Summer School for Evidence Based Public Health 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact The idea for this Summer School was conceived during an international workshop on "Evidence-Based Public Health: Concepts and Methods", held at the Ludwig-Maximilians-Universität (LMU) Munich in November 2010. Following on from direct and indirect feedback from this workshop and through consultation with international colleagues working in public health and evidence synthesis, a distinct requirement for training in evidence-based public health was identified. In response to this, the European Summer School in Evidence-Based Public Health has evolved as a collaboration between the LMU Munich (Dr Eva Rehfuess) and the University of Liverpool (Dr Daniel Pope). Including eminent speakers in the field of Evidence-Based Public Health, the Summer School takes place on a regular basis in either Liverpool or Munich.

The European Summer School on Evidence-Based Public Health has the following objectives:

• To learn about the principles of the evolving field of evidence-based public health through the use of worked examples.
• To embrace concepts of "complex interventions" and "complex systems" and to understand how these concepts affect evaluation of public health interventions in primary research and evidence synthesis.
• To understand the process, conduct and dissemination of systematic reviews in public health, focusing on quantitative reviews but being aware of qualitative/mixed-method and theory reviews.
• To get to know the GRADE approach as a means of assessing the quality of a body of evidence.
• To appreciate challenges of and different approaches to translating evidence into public health into policy and practice.

The Summer School includes:

• Keynote presentations from European experts in the field of evidence-based public health, followed by group discussions.
• Instruction in the conduct of systematic reviews and meta-analysis using examples that cover a variety of public health areas.
• Small group exercises and practice sessions in conceptualisation and evaluation of complex interventions, critical appraisal of published studies, meta-analysis and the application of the GRADE approach.
Year(s) Of Engagement Activity 2018