GRADE extension for complex social interventions

It is now standard practice for policy makers to draw on systematic reviews as the superior source of evidence to inform decision-making regarding effective interventions. One of the crucial steps in systematic review conduct and recommendation formulation involves a synthesis and rating of the quality of research evidence to help determine its validity and relevance for practice. However, social interventions, commonly applied within the disciplines of International Development, Psychology, Education, Criminology, Public Health, Social Work and Welfare are often complex. This means they might involve a number of interacting components, multiple outcomes, and diverse delivery formats. In addition, they might be more amenable to contextual factors and intervention implementation issues. For the research synthesis endeavours to be effective in guiding policy and practice, they must utilise adequate methodology that reflects the unique features of these interventions. Without this, research conclusions may be incorrect rendering policy recommendations erroneous and interventions inefficient.
The most prominent system to guide evidence-informed decision-making has been developed by the Grading of Recommendations Assessment, Development and Evaluation Working Group (GRADE) in clinical medicine. The GRADE approach provides a systematic and transparent process for rating the "best-available evidence" to inform recommendations for practice. Having been endorsed by more than 80 organisations worldwide, including the Cochrane Collaboration and the World Health Organisation, the applicability of the GRADE approach outside of clinical practice has been questioned, in part because of its neglect of important considerations of complex social interventions. Results from our previous investigation on the application of the GRADE approach in complex interventions indicate that GRADE may have limitations for these interventions. By way of illustration, GRADE initially starts with "low" quality rating for all types of observational studies; meanwhile, many social and health policy interventions are not possible to evaluate other than through observational and quasi-experimental study designs. This may lead to misinterpretations of evidence when transferred into practice, and therefore discourage important decisions. Furthermore, GRADE does not integrate important considerations in evaluation of these interventions, such as the implementation of an intervention, and integration of different types of the evidence. This suggests the need for a new methodological framework to summarise and rate the quality of evidence to inform decision-making in social disciplines.
The proposed project will develop and disseminate a framework for rating the quality of evidence and the strength of recommendations in complex social interventions. This new framework will be an official extension to the GRADE approach. For the purposes of guidance, a preliminary Steering Committee of leading experts and evaluation specialists across social disciplines has been established. With their coordination, this project will first organise an international online panel involving multidisciplinary expertise to generate a list of evidence quality definitions and criteria that will reflect the particular aspects of social interventions. Prior to the panel, a systematic review of the existing quality assessment tools and evidence grading systems will be undertaken with a specific focus on these interventions to elucidate the main areas of target. Following the online expert panel, a consensus meeting will be hosted with a select group of participants to generate the final list of definitions and criteria for the new framework. The generated GRADE extension will be published in high-impact social science journals, disseminated online, as well as in academic and professional conferences and meetings.

Apart from the impact on the academic and research sectors as described in the "Academic Beneficiaries" section, this project has a range of beneficiaries within the wider public sector and policy-making sphere both in low- and middle-income countries, and high-income countries.
Initially, the GRADE project was launched with the aim of providing a structured and transparent framework to guide healthcare management decisions and the processes of guideline development. Consequently, the GRADE extension for complex social interventions will promote transparency and consistency in the decision-making and organisation of care in a wide array of disciplines outside of clinical medicine, including International Development, Public Health, Psychology, Education, Criminology, Social Work and Welfare. By way of illustration, through the application of the generated framework agencies like Department of International Development, World Health Organisation (WHO), National Institute of Health and Care Excellence (NICE) public health and social care guidelines, Social Care Institute for Excellence (SCIE), Substance Abuse and Mental Health Services Administration (SAMHSA) among other others, that synthesise research evidence to provide guidance for effective social and healthcare interventions, will have appropriate methodology to summarise and rate the quality of evidence for these interventions and thus determine and adequately communicate their true value for practice. In addition, research organisations such as the Cochrane and Campbell Collaborations, International Initiative for Impact Evaluation (3ie), and Agency for Healthcare Research and Quality (AHRQ) will benefit from this framework in their aims of promoting high-quality research evidence syntheses regarding the effectiveness of complex psychosocial, public health, educational and development interventions. This will in turn better inform practice guidelines, policy-making and future research, and thus increase the efficiency of public services and social and health policies in a long run. In addition, this project will help optimise and enhance the cost-effectiveness of government investments when financing the design and implementation of different social, health promotion and economic development interventions to enhance social and health outcomes both in the UK, and internationally.
The GRADE extension for social interventions will allow policy-makers and practitioners to apply research evidence more consistently and explicitly in their decision-making. As already mentioned, social welfare will improve, because of enhanced rigour of empirical evidence for service providers to consult when offering various interventions. Meanwhile, consumers of interventions will have indirect benefits because of the augmented services tailored to their specific needs and circumstances. Dissemination of evidence-based practices and policy-making across core social disciplines will significantly increase health, quality of life and social wellbeing of all populations served by these interventions.