Empirical investigation of sources of bias in RCTs, using a new Cochrane bias assessment tool

Lead Research Organisation: University of Bristol
Department Name: Social Medicine

Abstract

Systematic reviews of randomised clinical trials provide the best evidence on the benefits and harms of medical treatments. However, the validity of their results depends on the included trials being conducted with methods that avoid bias. It is therefore standard practice to assess the risk of bias in the results of trials included in a systematic review. Collections of trials included in a large number of systematic reviews can be used to provide empirical evidence of bias in the results of trials.

The Cochrane Collaboration publishes about 300 new systematic reviews each year and aims to set the gold standard for their conduct. The Collaboration is launching a new tool for assessment of the risk of bias in trials. This project will evaluate the practical implementation of this tool. A database of 366 systematic reviews containing 3500 trials will be extended to include data from new Cochrane reviews. We will estimate associations of six types of bias with trial results.

The findings will contribute to improved interpretation of evidence about medical treatments. The results will be used as a basis for recommendations on the conduct and reporting of future trials, and hence ultimately to improved evidence for medicine.

Technical Summary

Background. Systematic reviews are key in the provision of evidence for clinical decision-making. As the validity of their results depends on that of the included randomised controlled trials (RCTs), it is essential that the methodological quality of the included RCTs is assessed, with bias assessments incorporated into the analysis and interpretation of reviews. The applicant, in her work on the BRANDO study, has combined data from ten empirical studies of the impact of bias on the results of RCTs into a single database containing data from 366 reviews and 3500 RCTs. The Cochrane Collaboration is launching a new tool for the assessment of the risk of bias in RCTs.

Aims. To evaluate the practical implementation of the Cochrane Collaboration‘s new Risk of Bias Tool (ROBT) and to conduct novel empirical research examining the effect of different types of bias on the results of RCTs.

Objectives.
1. To evaluate the implementation of the ROBT by Cochrane reviewers.
2. To extend the BRANDO database by incorporating information on new Cochrane reviews including studies evaluated using the ROBT.
3. To examine associations of high or unclear (compared to low) risk of bias, with intervention effect estimates, for each bias domain included in the ROBT, and in meta-analysis with different characteristics.
4. To compare the effects of bias observed in studies assessed with the new ROBT with effects of bias observed in previous empirical studies combined in the BRANDO database.

Design and Methodology. Qualitative research methods will be used to evaluate the practical implementation of the ROBT by Cochrane reviewers. The existing BRANDO database will be expanded to include new Cochrane reviews. A meta-epidemiological approach will be used to examine associations of risk of bias with intervention effect estimates. The effects of bias observed in studies assessed with the new ROBT will be compared with those observed in previous empirical studies combined in thBRANDO database.

Scientific and medical opportunities of the study.
This project will evaluate the impact of flaws in the conduct of recently published trials on intervention effect estimates. Its results will be used to adapt and update the ROBT, and inform guidelines on the conduct and reporting of RCTs. Ultimately, the project will contribute to improved interpretation of evidence about the effect of medical interventions, the improved conduct and reporting of future RCTs, and hence to better medical care.

Publications

10 25 50

publication icon
Savovic J (2010) Development of a combined database for meta-epidemiological research. in Research synthesis methods

publication icon
Savovic J (2018) Financial conflicts of interest in clinical research. in Intensive care medicine

 
Description MRC Hubs for Trial Methodology Research - Impact Award
Amount £9,972 (GBP)
Funding ID MR/L004933/2- N61 
Organisation University of Liverpool 
Sector Academic/University
Country United Kingdom
Start 11/2018 
End 10/2019
 
Description MRC Hubs for Trial Methodology Research - Network project funding and workshop grants
Amount £50,000 (GBP)
Funding ID MR/L004933/1- N61 
Organisation Medical Research Council (MRC) 
Department MRC Hubs for Trial Methodology Research
Sector Public
Country United Kingdom
Start 05/2015 
End 05/2016
 
Description MRC Methodology Panel
Amount £283,732 (GBP)
Funding ID MR/K014587/1 
Organisation Medical Research Council (MRC) 
Sector Academic/University
Country United Kingdom
Start 07/2013 
End 06/2016
 
Description MRC Methodology Panel
Amount £61,954 (GBP)
Funding ID MR/K01465X/1 
Organisation Medical Research Council (MRC) 
Sector Academic/University
Country United Kingdom
Start 05/2013 
End 01/2014
 
Description MRC Methodology Research Panel
Amount £367,999 (GBP)
Funding ID MR/M025209/1 
Organisation Medical Research Council (MRC) 
Sector Academic/University
Country United Kingdom
Start 10/2015 
End 12/2017
 
Description The Cochrane Opportunities Fund
Amount £29,534 (GBP)
Organisation The Cochrane Collaboration 
Department UK Cochrane Centre
Sector Charity/Non Profit
Country United Kingdom
Start 10/2009 
End 09/2010
 
Description the Cochrane Collaboration Methods Innovation Fund
Amount £69,967 (GBP)
Organisation The Cochrane Collaboration 
Department UK Cochrane Centre
Sector Charity/Non Profit
Country United Kingdom
Start 03/2012 
End 12/2014
 
Title ROBES database 
Description ROBES database of meta-analyses with risk of bias assessments for meta-epidemiological research. 
Type Of Material Database/Collection of data 
Year Produced 2013 
Provided To Others? Yes  
Impact ROBES database is used in the COMBAT study and for 2 PhD projects (one with and external collaborator and one in house). 
 
Description COMBAT study: Development of a method for adjusting trial results for biases in meta-analysis: combining generic evidence on bias with detailed trial assessment 
Organisation University of Cambridge
Department MRC Biostatistics Unit
Country United Kingdom 
Sector Public 
PI Contribution We contributed to drafting the grant proposal, which was successful. We started assembling data from July 2013.
Collaborator Contribution The partner (PI) approached us to collaborate to combine their proposed method with ours and wrote the first draft of the grant proposal. They are currently advising on the methods and will have more hands on input at the analysis stage.
Impact Funding awarded by MRC in December 2012. Project is completed, with 2 papers published, one submitted and one in preparation (as of Feb 2019). Disciplines: Epidemiology, medical statistics, evidence synthesis, opinion elicitation.
Start Year 2011
 
Description Data mining approach to assessment of risk of bias in RCTs from published trial reports (PhD project) 
Organisation University of Bristol
Country United Kingdom 
Sector Academic/University 
PI Contribution Provided ROBES database and acted as adviser in the study design stages of the PhD project.
Collaborator Contribution PhD student working on the project.
Impact Student completed their PhD. They had a paper in which they acknowledged they used data from ROBES study. Project was multidisciplinary, involving the following disciplines: evidence synthesis (including meta-analysis, statistics and epidemiology), and data-mining/ text-mining / programming /computer science.
Start Year 2012
 
Description Developing Latitudes Network - an online Library of Assessment Tools and InsTruments Used to assess Data validity in Evidence Synthesis 
Organisation Equator Network
Country United Kingdom 
Sector Charity/Non Profit 
PI Contribution This is a collaboration at an early stage. Our core steering group approached the steering group members of the Equator Network to help us build Latitudes Network online resources in partnership with them. Latitudes Network will be complementary to the Equator Network and provide an accessible online resource for quality and bias assessment tools for health care research. We have planned the initial meeting to plan a bid to fund this initiative.
Collaborator Contribution This at an early stage. Partners will provide support for the project and will help us with their Experience with Equator Network.
Impact We had several unsuccessful attempts to obtain funding. The first paper from this collaboration was published in J Clin Epidemiol in 2017. The collaboration is still active but it is difficult to push ahead without dedicated funding.
Start Year 2013
 
Description Developing Latitudes Network - an online Library of Assessment Tools and InsTruments Used to assess Data validity in Evidence Synthesis 
Organisation Kleijnen Systematic Reviews Ltd
Country United Kingdom 
Sector Private 
PI Contribution This is a collaboration at an early stage. Our core steering group approached the steering group members of the Equator Network to help us build Latitudes Network online resources in partnership with them. Latitudes Network will be complementary to the Equator Network and provide an accessible online resource for quality and bias assessment tools for health care research. We have planned the initial meeting to plan a bid to fund this initiative.
Collaborator Contribution This at an early stage. Partners will provide support for the project and will help us with their Experience with Equator Network.
Impact We had several unsuccessful attempts to obtain funding. The first paper from this collaboration was published in J Clin Epidemiol in 2017. The collaboration is still active but it is difficult to push ahead without dedicated funding.
Start Year 2013
 
Description Developing Latitudes Network - an online Library of Assessment Tools and InsTruments Used to assess Data validity in Evidence Synthesis 
Organisation University of Birmingham
Country United Kingdom 
Sector Academic/University 
PI Contribution This is a collaboration at an early stage. Our core steering group approached the steering group members of the Equator Network to help us build Latitudes Network online resources in partnership with them. Latitudes Network will be complementary to the Equator Network and provide an accessible online resource for quality and bias assessment tools for health care research. We have planned the initial meeting to plan a bid to fund this initiative.
Collaborator Contribution This at an early stage. Partners will provide support for the project and will help us with their Experience with Equator Network.
Impact We had several unsuccessful attempts to obtain funding. The first paper from this collaboration was published in J Clin Epidemiol in 2017. The collaboration is still active but it is difficult to push ahead without dedicated funding.
Start Year 2013
 
Description Developing Latitudes Network - an online Library of Assessment Tools and InsTruments Used to assess Data validity in Evidence Synthesis 
Organisation University of Oxford
Department Nuffield Department of Primary Care Health Sciences
Country United Kingdom 
Sector Academic/University 
PI Contribution This is a collaboration at an early stage. Our core steering group approached the steering group members of the Equator Network to help us build Latitudes Network online resources in partnership with them. Latitudes Network will be complementary to the Equator Network and provide an accessible online resource for quality and bias assessment tools for health care research. We have planned the initial meeting to plan a bid to fund this initiative.
Collaborator Contribution This at an early stage. Partners will provide support for the project and will help us with their Experience with Equator Network.
Impact We had several unsuccessful attempts to obtain funding. The first paper from this collaboration was published in J Clin Epidemiol in 2017. The collaboration is still active but it is difficult to push ahead without dedicated funding.
Start Year 2013
 
Description Developing ROBIS - a new tool to assess the risk of bias in systematic reviews 
Organisation Kleijnen Systematic Reviews Ltd
Country United Kingdom 
Sector Private 
PI Contribution We designed the project and successfully applied for funding . I co-led the preliminary review work and produced relevant materials to be considered at the development meeting. I organised the international meeting of experts where we developed the first draft of the risk of bias tool. I am currently co-leading the refinement and making arrangements for piloting
Collaborator Contribution Project design, application for funding, review, participation in the development meeting, refinement. We anticipate that some of the partners will contribute to piloting.
Impact We established collaborations with many international organisations and experts who contributet to the development and piloting of the ROBIS tool, which was published in 2015 Paper: Whiting P, Savovic J, Higgins JP, Caldwell DM, Reeves BC, Shea B, Davies P, Kleijnen J, Churchill R; ROBIS group. ROBIS: A new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol. 2016 Jan;69:225-34. doi: 10.1016/j.jclinepi.2015.06.005. This paper was translated to Spanish and Italian. Presentations: We held numerous training workshops of the tool at Cochrane and other meetings Impact: We have provided training on ROBIS tool to some of the NICE guidelines development teams and to the Estonian Health Insurance Agency, who plan to use the tool for guidelines development. It is also one of the Cochrane-recommended tools for use in overviews.
Start Year 2012
 
Description Developing ROBIS - a new tool to assess the risk of bias in systematic reviews 
Organisation The Cochrane Collaboration
Department Cochrane Bias Methods Group
Country Canada 
Sector Academic/University 
PI Contribution We designed the project and successfully applied for funding . I co-led the preliminary review work and produced relevant materials to be considered at the development meeting. I organised the international meeting of experts where we developed the first draft of the risk of bias tool. I am currently co-leading the refinement and making arrangements for piloting
Collaborator Contribution Project design, application for funding, review, participation in the development meeting, refinement. We anticipate that some of the partners will contribute to piloting.
Impact We established collaborations with many international organisations and experts who contributet to the development and piloting of the ROBIS tool, which was published in 2015 Paper: Whiting P, Savovic J, Higgins JP, Caldwell DM, Reeves BC, Shea B, Davies P, Kleijnen J, Churchill R; ROBIS group. ROBIS: A new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol. 2016 Jan;69:225-34. doi: 10.1016/j.jclinepi.2015.06.005. This paper was translated to Spanish and Italian. Presentations: We held numerous training workshops of the tool at Cochrane and other meetings Impact: We have provided training on ROBIS tool to some of the NICE guidelines development teams and to the Estonian Health Insurance Agency, who plan to use the tool for guidelines development. It is also one of the Cochrane-recommended tools for use in overviews.
Start Year 2012
 
Description Developing ROBIS - a new tool to assess the risk of bias in systematic reviews 
Organisation The Cochrane Collaboration
Country Global 
Sector Charity/Non Profit 
PI Contribution We designed the project and successfully applied for funding . I co-led the preliminary review work and produced relevant materials to be considered at the development meeting. I organised the international meeting of experts where we developed the first draft of the risk of bias tool. I am currently co-leading the refinement and making arrangements for piloting
Collaborator Contribution Project design, application for funding, review, participation in the development meeting, refinement. We anticipate that some of the partners will contribute to piloting.
Impact We established collaborations with many international organisations and experts who contributet to the development and piloting of the ROBIS tool, which was published in 2015 Paper: Whiting P, Savovic J, Higgins JP, Caldwell DM, Reeves BC, Shea B, Davies P, Kleijnen J, Churchill R; ROBIS group. ROBIS: A new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol. 2016 Jan;69:225-34. doi: 10.1016/j.jclinepi.2015.06.005. This paper was translated to Spanish and Italian. Presentations: We held numerous training workshops of the tool at Cochrane and other meetings Impact: We have provided training on ROBIS tool to some of the NICE guidelines development teams and to the Estonian Health Insurance Agency, who plan to use the tool for guidelines development. It is also one of the Cochrane-recommended tools for use in overviews.
Start Year 2012
 
Description Developing ROBIS - a new tool to assess the risk of bias in systematic reviews 
Organisation University of Bristol
Department School of Clinical Sciences
Country United Kingdom 
Sector Academic/University 
PI Contribution We designed the project and successfully applied for funding . I co-led the preliminary review work and produced relevant materials to be considered at the development meeting. I organised the international meeting of experts where we developed the first draft of the risk of bias tool. I am currently co-leading the refinement and making arrangements for piloting
Collaborator Contribution Project design, application for funding, review, participation in the development meeting, refinement. We anticipate that some of the partners will contribute to piloting.
Impact We established collaborations with many international organisations and experts who contributet to the development and piloting of the ROBIS tool, which was published in 2015 Paper: Whiting P, Savovic J, Higgins JP, Caldwell DM, Reeves BC, Shea B, Davies P, Kleijnen J, Churchill R; ROBIS group. ROBIS: A new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol. 2016 Jan;69:225-34. doi: 10.1016/j.jclinepi.2015.06.005. This paper was translated to Spanish and Italian. Presentations: We held numerous training workshops of the tool at Cochrane and other meetings Impact: We have provided training on ROBIS tool to some of the NICE guidelines development teams and to the Estonian Health Insurance Agency, who plan to use the tool for guidelines development. It is also one of the Cochrane-recommended tools for use in overviews.
Start Year 2012
 
Description Developing ROBIS - a new tool to assess the risk of bias in systematic reviews 
Organisation University of Ottawa
Country Canada 
Sector Academic/University 
PI Contribution We designed the project and successfully applied for funding . I co-led the preliminary review work and produced relevant materials to be considered at the development meeting. I organised the international meeting of experts where we developed the first draft of the risk of bias tool. I am currently co-leading the refinement and making arrangements for piloting
Collaborator Contribution Project design, application for funding, review, participation in the development meeting, refinement. We anticipate that some of the partners will contribute to piloting.
Impact We established collaborations with many international organisations and experts who contributet to the development and piloting of the ROBIS tool, which was published in 2015 Paper: Whiting P, Savovic J, Higgins JP, Caldwell DM, Reeves BC, Shea B, Davies P, Kleijnen J, Churchill R; ROBIS group. ROBIS: A new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol. 2016 Jan;69:225-34. doi: 10.1016/j.jclinepi.2015.06.005. This paper was translated to Spanish and Italian. Presentations: We held numerous training workshops of the tool at Cochrane and other meetings Impact: We have provided training on ROBIS tool to some of the NICE guidelines development teams and to the Estonian Health Insurance Agency, who plan to use the tool for guidelines development. It is also one of the Cochrane-recommended tools for use in overviews.
Start Year 2012
 
Description Development of methods for dealing with selective non-reporting of trial results 
Organisation Monash University
Country Australia 
Sector Academic/University 
PI Contribution We are hosting a visiting senior research associate from Monash University for 2 years. The main topic of their fellowship is Development of methods for dealing with selective non-reporting of trial results. Some of the work on the ROBES study and the data from the ROBES database were used by the fellow for preliminary stages of the collaborative research.
Collaborator Contribution Visiting senior research associate from Monash University has obtained their own funding to fund their 2-year visit in our institution. The visiting postdoc is currently collaborating with our team on several related projects.
Impact No outputs yet.
Start Year 2015
 
Description Evaluation of the Cochrane Collaboration's Risk of Bias Tool 
Organisation The Cochrane Collaboration
Department Cochrane Bias Methods Group
Country Canada 
Sector Academic/University 
PI Contribution I wrote the first draft of the grant application and then liaised with the collaborators to produce the final version of the application based on all collaborators' input. Lead applicant (PI) for this grant application is my MRC Training Fellowship Sponsor. I worked as a lead researcher on this project (as a part of a 6-month abeyance from my MRC fellowship) from November 2009 to April 2010. The evaluation is now finished, but all of the collaborators are still advising and overseeing the implementation of the recommended changes to the Risk of Bias Tool that came out from the evaluation process.
Collaborator Contribution All collaborators participated in writing the grant proposal and successfully applying for funding to The Cochrane Collaboration's Opportunities Fund. After the award all collaborators continue to advise on the project.
Impact 1. Funding obtained for the project: £29,354 received by my institution from the Cochrane Opportunities Fund. 2. Publication in the BMJ, a part of which is based on the results from this collaboration. (see publication section for citation details). 3. Publication in Systematic Reviews (2014) 4. This project has led to new collaborations/initiatives: - Extending Cochrane Risk of Bias tool for non-randomised studies - Revisions of the Cochrane Risk of Bias tool for RCTs
Start Year 2009
 
Description Extending the Cochrane Collaboration's Risk of Bias tool for non-randomised studies and non-standard trial designs 
Organisation The Cochrane Collaboration
Country Global 
Sector Charity/Non Profit 
PI Contribution I drfated a grant proposal for this methodological development and applied for funding to the Cochrane Collaboration, together with co-applicants from the Cochrane Bias Methods Group and Non-randomised Studies Methods Group and collaborators from other Cochrane entities.
Collaborator Contribution Partners contribute in all stages of the development of the tool and piloting.
Impact Successful grant proposal to the Cochrane Collaboration Methods Innovation Fund. The collaboration consisted of research methodologists, epidemiologists, statisticians, editors, and other Cochrane stake-holders. Publication: Sterne et al. BMJ. 2016 Oct 12;355:i4919. doi: 10.1136/bmj.i4919. This has led to a further grant from the MRC by Julian Higgins (PI) and collaborators to further develop the tool for different types of study designs. ROBINS-I tool will be included as a chapter in the new version of the Cochrane Handbook (available in in 2019)
Start Year 2011
 
Description Extending the Cochrane Collaboration's Risk of Bias tool for non-randomised studies and non-standard trial designs 
Organisation The Cochrane Collaboration
Country Global 
Sector Charity/Non Profit 
PI Contribution I drfated a grant proposal for this methodological development and applied for funding to the Cochrane Collaboration, together with co-applicants from the Cochrane Bias Methods Group and Non-randomised Studies Methods Group and collaborators from other Cochrane entities.
Collaborator Contribution Partners contribute in all stages of the development of the tool and piloting.
Impact Successful grant proposal to the Cochrane Collaboration Methods Innovation Fund. The collaboration consisted of research methodologists, epidemiologists, statisticians, editors, and other Cochrane stake-holders. Publication: Sterne et al. BMJ. 2016 Oct 12;355:i4919. doi: 10.1136/bmj.i4919. This has led to a further grant from the MRC by Julian Higgins (PI) and collaborators to further develop the tool for different types of study designs. ROBINS-I tool will be included as a chapter in the new version of the Cochrane Handbook (available in in 2019)
Start Year 2011
 
Description How much heterogeneity in meta-analyses is explained by bias in included trials? (PhD project) 
Organisation University of Cambridge
Department MRC Biostatistics Unit
Country United Kingdom 
Sector Public 
PI Contribution I have contributed data from the ROBES study for a PhD project and acted as an external adviser on this methodological PhD project.
Collaborator Contribution The Biostatistics Unit PhD student worked on this new project.
Impact The project is completed (student completed their PhD successfully) and the paper published in a peer review journal. The project is in the field statistics in meta-analysis and meta-epidemiology of bias in trials, so not multidisciplinary.
Start Year 2013
 
Description MetaBlind study 
Organisation Copenhagen University Hospital
Department Nordic Cochrane Centre
Country Denmark 
Sector Academic/University 
PI Contribution We wrote a grant proposal for funding together with the partners.
Collaborator Contribution One of the partners did some pilot work to inform the main research proposal. They contributed a great deal to writing the proposal.
Impact We submitted a grant proposal to MRC Methodology Panel in June 2013 and this was rejected. Our Danish partners have found funding in Denmark and the majority of this project has been carried out in Denmark. Analyses have been carried out at our institution (by a PhD student) and the manuscripts have been submitted to peer review journals.
Start Year 2012
 
Description MetaBlind study 
Organisation University of Paris - Descartes
Country France 
Sector Academic/University 
PI Contribution We wrote a grant proposal for funding together with the partners.
Collaborator Contribution One of the partners did some pilot work to inform the main research proposal. They contributed a great deal to writing the proposal.
Impact We submitted a grant proposal to MRC Methodology Panel in June 2013 and this was rejected. Our Danish partners have found funding in Denmark and the majority of this project has been carried out in Denmark. Analyses have been carried out at our institution (by a PhD student) and the manuscripts have been submitted to peer review journals.
Start Year 2012
 
Description Conference presentation of ROBES study results, Seoul 2016 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Oral presentation at a conference (20min), presented to ~50 international researchers: Savovic J. A new large-scale meta-epidemiological study on bias in randomized trials using routinely collected 'Risk of bias' assessments by Cochrane authors: results from the ROBES study. Cochrane Colloquium, Seoul, South Korea, 23-27 October 2016.
Year(s) Of Engagement Activity 2016
 
Description Conference presentation of RoB 2 at ICTMC/SCT joint conference, Liverpool 2017 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Savovic J, Page M, Elbers R, Hrobjartsson A, Boutron I, Reeves B, Sterne J, Higgins J. A revised tool for assessing risk of bias in randomized trials (RoB 2.0). Joint 38th Society for Clinical trials and 4th International Clinical Trials Methodology Conference, Liverpool, UK, 7th-10th May 2017.
Oral presentation at a conference (15 min) to about ~100 international delegates.
Year(s) Of Engagement Activity 2017
 
Description Conference presentation: Evaluation of the Cochrane Risk of Bias tool. Cochrane Colloquium, Keystone 2010. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Oral presentation at a conference attended by international researchers: Savovic J, Altman D, Higgins J, Moher D, Sterne J, Turner L, Weeks L. Risk of bias tool evaluation: process and results. 18th Cochrane Colloquium, Keystone, Colorado, USA, Oct 2010.
Year(s) Of Engagement Activity 2010
 
Description Invited 3h workshop on RoB 2 at Cochrane Colloquium, Seoul 2016 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Higgins J, Sterne J, Savovic J. Introducing a revised tool for assessing risk of bias in randomized trials. Cochrane Colloquium, Seoul, South Korea, 23-27 October 2016. Invited special 3-hour pre-conference tarining workshop for ~100 international researchers.
Year(s) Of Engagement Activity 2016
 
Description Invited special seminar: Announcement of revised tools to assess risk of bias in randomized and non-randomized studies, Cochrane Colloquium, Seoul 2016 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Higgins J, Sterne J, Savovic J. Announcement of revised tools to assess risk of bias in randomized trials and in non-randomized studies. Cochrane Colloquium, Seoul, South Korea, 23-27 October 2016. Invited special seminar (50 min duration) to over 100 international researchers.
Year(s) Of Engagement Activity 2016
 
Description Oral presentation: RoB 2 at the Cochrane BMG AGM at Cochrane Colloquium, Seoul 2016 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Invited short talk about the revised RoB 2 tool for the Cochrane Methods Group members.
Savovic J. Introducing a revised tool for assessing risk of bias in randomized trials. Cochrane Bias Methods Group Annual Meeting; Cochrane Colloquium, Seoul, South Korea, 23-27 October 2016.
Year(s) Of Engagement Activity 2016
 
Description RoB 2 workshops at Cochrane Colloquium, Edinburgh 2018 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Savovic J, MacAleenan A, Sterne J, Higgins J. Assessing risk of bias in randomized trials (RoB 2). Cochrane Colloquium, Edinburgh, Scotland, 16-18 September 2018.
Two workshops (delivered twice during the conference due to high demand) training researchers from across the world to use our new risk of bias tool for RCTs (RoB 2). Attended by ~160 participants in total over the two workshops.
Year(s) Of Engagement Activity 2018
 
Description Short presentation: Empirical evidence of bias, Launch of the Bristol Population Health Science Institute, Bristol 2017 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact Oral presentation at the launch event of the new institute at the University of Bristol:
Savovic J. Empirical evidence of bias via meta-epidemiology. Launch of the Bristol Population Health Science Institute. Bristol, UK, 9 June 2017.
Year(s) Of Engagement Activity 2017
 
Description Workshop for early version of RoB 2 at Cochrane Colloquium, Vienna 2015 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Workshop introducing the first version of the revised risk of bias tool for RCTs (RoB 2.0) to ~100 international researchers.
Savovic J, Higgins JPT, Sterne JAC, Boutron I, Hrobjartsson A. Introducing a revised risk of bias tool for randomized trials. 23rd Cochrane Colloquium, Vienna, Austria, 3-7 Oct 2015.
Year(s) Of Engagement Activity 2015
 
Description Workshop: Evaluation of the Cochrane Risk of Bias tool. Cochrane Colloquium, Keystone 2010. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Large workshop for 80-100 international researcher and Cochrane editors on the findings of the evaluation of the Cochrane RoB tool introduced in 2008 and discussion of plans for the tool's future updates and revisions.
Sterne J, Savovic J, Turner L, Altman D, Higgins J, Moher D, Weeks L. Findings and recommendations from the evaluation of the Cochrane Collaboration's Risk of Bias assessment tool. 18th Cochrane Colloquium, Keystone, Colorado, USA, Oct 2010.
Year(s) Of Engagement Activity 2010
 
Description Workshop: update on the extension to Cochrane ROB tool to NRS and Cluster RCTs at Cochrane Colloquium, Quebec 2013 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Workshop introducing work in progress on the development of the new risk of bias tool for non-randomized studies and the revision of the Cochrane risk of bias tool for RCTs. Attended by 80-100 international researchers.
Sterne JAC, Higgins JPT, Reeves BC, Savovic J, Turner L. Extending the risk of bias tool to allow for assessment of non-randomised studies, cluster-randomised trials and cross-over trials: a Cochrane methods innovation fund project. 21st Cochrane Colloquium, Québec, Canada, 19-23 Sep 2013.
Year(s) Of Engagement Activity 2013