Empirical investigation of sources of bias in RCTs, using a new Cochrane bias assessment tool
Lead Research Organisation:
University of Bristol
Department Name: Social Medicine
Abstract
Systematic reviews of randomised clinical trials provide the best evidence on the benefits and harms of medical treatments. However, the validity of their results depends on the included trials being conducted with methods that avoid bias. It is therefore standard practice to assess the risk of bias in the results of trials included in a systematic review. Collections of trials included in a large number of systematic reviews can be used to provide empirical evidence of bias in the results of trials.
The Cochrane Collaboration publishes about 300 new systematic reviews each year and aims to set the gold standard for their conduct. The Collaboration is launching a new tool for assessment of the risk of bias in trials. This project will evaluate the practical implementation of this tool. A database of 366 systematic reviews containing 3500 trials will be extended to include data from new Cochrane reviews. We will estimate associations of six types of bias with trial results.
The findings will contribute to improved interpretation of evidence about medical treatments. The results will be used as a basis for recommendations on the conduct and reporting of future trials, and hence ultimately to improved evidence for medicine.
The Cochrane Collaboration publishes about 300 new systematic reviews each year and aims to set the gold standard for their conduct. The Collaboration is launching a new tool for assessment of the risk of bias in trials. This project will evaluate the practical implementation of this tool. A database of 366 systematic reviews containing 3500 trials will be extended to include data from new Cochrane reviews. We will estimate associations of six types of bias with trial results.
The findings will contribute to improved interpretation of evidence about medical treatments. The results will be used as a basis for recommendations on the conduct and reporting of future trials, and hence ultimately to improved evidence for medicine.
Technical Summary
Background. Systematic reviews are key in the provision of evidence for clinical decision-making. As the validity of their results depends on that of the included randomised controlled trials (RCTs), it is essential that the methodological quality of the included RCTs is assessed, with bias assessments incorporated into the analysis and interpretation of reviews. The applicant, in her work on the BRANDO study, has combined data from ten empirical studies of the impact of bias on the results of RCTs into a single database containing data from 366 reviews and 3500 RCTs. The Cochrane Collaboration is launching a new tool for the assessment of the risk of bias in RCTs.
Aims. To evaluate the practical implementation of the Cochrane Collaboration‘s new Risk of Bias Tool (ROBT) and to conduct novel empirical research examining the effect of different types of bias on the results of RCTs.
Objectives.
1. To evaluate the implementation of the ROBT by Cochrane reviewers.
2. To extend the BRANDO database by incorporating information on new Cochrane reviews including studies evaluated using the ROBT.
3. To examine associations of high or unclear (compared to low) risk of bias, with intervention effect estimates, for each bias domain included in the ROBT, and in meta-analysis with different characteristics.
4. To compare the effects of bias observed in studies assessed with the new ROBT with effects of bias observed in previous empirical studies combined in the BRANDO database.
Design and Methodology. Qualitative research methods will be used to evaluate the practical implementation of the ROBT by Cochrane reviewers. The existing BRANDO database will be expanded to include new Cochrane reviews. A meta-epidemiological approach will be used to examine associations of risk of bias with intervention effect estimates. The effects of bias observed in studies assessed with the new ROBT will be compared with those observed in previous empirical studies combined in the BRANDO database.
Scientific and medical opportunities of the study.
This project will evaluate the impact of flaws in the conduct of recently published trials on intervention effect estimates. Its results will be used to adapt and update the ROBT, and inform guidelines on the conduct and reporting of RCTs. Ultimately, the project will contribute to improved interpretation of evidence about the effect of medical interventions, the improved conduct and reporting of future RCTs, and hence to better medical care.
Aims. To evaluate the practical implementation of the Cochrane Collaboration‘s new Risk of Bias Tool (ROBT) and to conduct novel empirical research examining the effect of different types of bias on the results of RCTs.
Objectives.
1. To evaluate the implementation of the ROBT by Cochrane reviewers.
2. To extend the BRANDO database by incorporating information on new Cochrane reviews including studies evaluated using the ROBT.
3. To examine associations of high or unclear (compared to low) risk of bias, with intervention effect estimates, for each bias domain included in the ROBT, and in meta-analysis with different characteristics.
4. To compare the effects of bias observed in studies assessed with the new ROBT with effects of bias observed in previous empirical studies combined in the BRANDO database.
Design and Methodology. Qualitative research methods will be used to evaluate the practical implementation of the ROBT by Cochrane reviewers. The existing BRANDO database will be expanded to include new Cochrane reviews. A meta-epidemiological approach will be used to examine associations of risk of bias with intervention effect estimates. The effects of bias observed in studies assessed with the new ROBT will be compared with those observed in previous empirical studies combined in the BRANDO database.
Scientific and medical opportunities of the study.
This project will evaluate the impact of flaws in the conduct of recently published trials on intervention effect estimates. Its results will be used to adapt and update the ROBT, and inform guidelines on the conduct and reporting of RCTs. Ultimately, the project will contribute to improved interpretation of evidence about the effect of medical interventions, the improved conduct and reporting of future RCTs, and hence to better medical care.
