Interesting paper on systematic reviews:
CITATION: Res Synth Methods. 2022 Mar 12. doi: 10.1002/jrsm.1556. Online ahead of print.
When is the evidence conclusive? Analysis of systematic reviews for which Cochrane declared that conclusions will not change with further studies.
Babić A(1), Poklepović Peričić T(2), Pieper D(3)(4), Puljak L(5).
https://onlinelibrary.wiley.com/doi/10.1002/jrsm.1556?af=R
'Some Cochrane reviews were declared stable or closed, that is, not in need of updating. For some of them, it has been declared that conclusions will not (or it is unlikely they will) change with further studies. We explored whether there is a discernable decision-making pattern for decisions about the conclusiveness and stabilization of these reviews. We analyzed Cochrane reviews published until April 2020 labeled as stable or closed. We extracted the rationale leading to the decision declaring that the conclusion is not expected to change with further studies. Furthermore, we assessed whether the reviews used GRADE analysis. We extracted data from summary of findings (SoF) tables on the direction of effect, statistical significance, and I2 values for the first and primary outcomes in SoFs, conclusions in the abstract and review, and implications for practice and future research. We included 40 stable/closed Cochrane reviews. Rationales for their stabilization did not enable any insight into the Cochrane's decision-making algorithm for considering the evidence as conclusive. Among 191 outcomes presented in the SoFs, 70% were rated with either low or very low certainty evidence. None of the reviews mentioned in the text that the review should be stabilized or closed, or that there is sufficient evidence on the subject. Reasons for stabilizing/closing Cochrane reviews were unclear, and we could not discern any pattern of "conclusive review" traits. Definition of systematic review conclusiveness is still lacking, which may contribute to research waste.'
Neil Pakenham-Walsh, HIFA Coordinator, neil@hifa.org www.hifa.org
