CONSORT guideline for experiments of psychological, social, and environmental

Experiments are essential for assessing the benefits and harms of psychological, social, and environmental interventions. However, these interventions can be costly and difficult to evaluate because they are often complex, involving multiple interacting components. For experiments to be replicable and transferable into policy and practice, authors must report intervention and study design, conduct, and analysis clearly. Without this detail, results may be unusable, syntheses may be impossible, and scarce resources may be wasted.
Reporting guidelines have improved the quality of reports in medical experiments, and thus their real-world utility. However, no evidence-based guidelines have been developed for reporting the unique features of experiments in criminology, education, psychology, public health, and social work. Publications of these studies often fail to provide adequate information for critical appraisal, replication of the intervention, or research synthesis. To maximise the investment made in such research, a new reporting guideline is needed.

This project will develop and disseminate a guideline for reporting experiments of psychological, social, and environmental interventions. To coordinate and publicise the initiative, we have assembled a Steering Group of leading experts in core social science disciplines. With their guidance, we will first conduct an international Delphi process with researchers, journal editors, and other stakeholders to generate a list of important reporting standards to consider for inclusion in the guideline. The Delphi process will identify items that are required to allow replication, assess validity, and transfer scientific knowledge to real-world settings. We have conducted two reviews that identify areas of reporting to target, in order to maximise the quality, acceptability, and uptake of a guideline across disciplines. Results indicate that publications often omit information that would allow readers to assess internal validity (e.g. blinding and level of randomisation). Such reports may overestimate intervention effects by as much as 30%.[17] Secondly, reports should include more information relevant to the external validity of experiments (e.g. participant selection, intervention implementation). To be useful for experiments in criminology, education, psychology, public health, and social work, the CONSORT Statement should be modified and extended. The Delphi process will identify those items most important for understanding the unique features of these studies.

We will then host a consensus meeting with a select group of Delphi participants to finalise the list of minimal reporting standards for inclusion in the guideline. Next, we will codify these standards into a user-friendly checklist, draft the guideline document, and write a companion paper to explain the rationale for each item and provide examples of good reporting.

Our dissemination strategy begins with stakeholder involvement in the design and execution of this project, ensuring that the guideline will be acceptable and widely endorsed by users. Once completed, the guideline and explanatory paper will be published simultaneously in multiple, high-impact journals. To promote pervasive and habitual use, we will seek endorsement of the guideline by psychological and social science journals. The guideline will be disseminated online, at conferences in all appropriate fields and through policy briefs. We will target researchers, educators, students, funders, practitioners and other consumers.

The outputs from this project will help authors write clear reports, create a framework for reviewers to assess publications, expedite funding evaluations, provide a pedagogical tool for understanding these experiments, and help consumers evaluate study validity and applicability. In these ways, the guideline will facilitate efficient and effective transfer of research evidence into real-world use.

Psychological, social, and environmental experiments often cost a significant amount of money. Poor reporting of experiments, like poor conduct of experiments, can render such studies practically useless. If this project succeeds in making just one large experiment usable for decision-making, its economic gain will already outweigh its cost.

This project has many societal and economic impacts. Academics and researchers will benefit as described in the 'Academic Beneficiaries' section. The public sector will benefit by having higher quality experimental evaluations and published reports of experiments to guide policy and practice decisions. For example, NICE guidelines for mental health care (and their corollaries in other countries), which systematically utilise the current research evidence to make recommendations, will have better reports with sufficient information to appraise the quality and validity of these experiments and determine their relevance for practice. Systematic review organisations, such as the Cochrane and Campbell Collaborations, will be better placed to synthesise research evidence regarding the effectiveness of psychological, social, and environmental interventions, and thereby provide better reviews for clinical guidelines, policy-making, and future research. Moreover, public sector departments, ministries, and funding agencies will have greater return on investment when financing psychological, social, and environmental intervention experiments.

Higher quality reports and trials could influence policy-makers and practitioners to utilise research evidence more consistently in their decision-making. Public services and policy would become more effective, as choices are more frequently informed by better evidence. Social welfare may improve as service providers and third sector organisations change their culture to offer programmes based on rigorous empirical evidence when possible. Service users will benefit indirectly as the interventions they receive will improve. In addition, as some consumers have begun to inform themselves about the treatments they receive, these guidelines may clarify the validity and applicability of psychological, social, and environmental interventions to consumers' conditions and situations. Increases in evidence-based practice and policy-making, both in the UK and internationally, could contribute significantly to increased health and quality of life for all populations served by these interventions.