RoB 2 guidance for all systematic reviewers from Cochrane pilot

The Cochrane Collaboration switched from using its first risk of bias tool to using the updated version, RoB 2, in 2019. Cochrane adopted a phased approach allowing authors to decide when to use it and offering them support from the editorial team. A paper outlining their experience and lessons learned has been published in BMJ Evidence-Based Medicine.

The 2019 relaunch aimed to improve its usability and to reflect current understanding of how the causes of bias can influence study results. While it is the recommended tool to assess the risk of bias in randomised control trials included in Cochrane Reviews, it is also  likely to be widely adopted by many systematic reviewers outside of Cochrane.

Cochrane’s phased approach meant users of the tool informed the development of guidance. The RoB 2 tool has some fundamental differences from the original tool, which was first used in 2008. This caused some issues for some author teams where the tool was not applied correctly.

The paper highlights learning for all systematic reviewers, Cochrane and non-Cochrane, on:

• The differences between the original version of the tool and RoB 2
• Key steps to take when planning a review using RoB 2 for reporting both protocols or full reviews

The guidance was all based on learning from the authors during the pilot phase of the implementation.

The paper’s authors, including the ROB 2 tool developers, Cochrane editors and ARC West’s Tess Moore and Julian Higgins, concluded:

“Assessing the risk of bias for specific results allows us to see the impact this has on a synthesis or meta-analysis, whereas previously a study was usually judged for bias overall and might have left some results with a worse assessment of bias than was warranted.”