Software tools to support title and abstract screening for systematic reviews in healthcare: an evaluation

19 January, 2020

(with thanks to Irina Ibraghimova and LRC Network)

'Based on this study, we would recommend Covidence and Rayyan to systematic reviewers looking for suitable and easy to use tools to support title and abstract screening within healthcare research'. The paper makes no reference to language or translation, suggesting that the tools are designed for English-language screening only. Also, will such tools help or paradoxically hinder efforts to increase the quality of systematic reviews?

CITATION: Software tools to support title and abstract screening for systematic reviews in healthcare: an evaluation

Hannah Harrison, Simon J. Griffin, Isla Kuhn & Juliet A. Usher-Smith

BMC Medical Research Methodology volume 20, Article number: 7 (2020)

https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-020-...

ABSTRACT

Background: Systematic reviews are vital to the pursuit of evidence-based medicine within healthcare. Screening titles and abstracts (T&Ab) for inclusion in a systematic review is an intensive, and often collaborative, step. The use of appropriate tools is therefore important. In this study, we identified and evaluated the usability of software tools that support T&Ab screening for systematic reviews within healthcare research.

Methods: We identified software tools using three search methods: a web-based search; a search of the online “systematic review toolbox”; and screening of references in existing literature. We included tools that were accessible and available for testing at the time of the study (December 2018), do not require specific computing infrastructure and provide basic screening functionality for systematic reviews. Key properties of each software tool were identified using a feature analysis adapted for this purpose. This analysis included a weighting developed by a group of medical researchers, therefore prioritising the most relevant features. The highest scoring tools from the feature analysis were then included in a user survey, in which we further investigated the suitability of the tools for supporting T&Ab screening amongst systematic reviewers working in medical research.

Results: Fifteen tools met our inclusion criteria. They vary significantly in relation to cost, scope and intended user community. Six of the identified tools (Abstrackr, Colandr, Covidence, DRAGON, EPPI-Reviewer and Rayyan) scored higher than 75% in the feature analysis and were included in the user survey. Of these, Covidence and Rayyan were the most popular with the survey respondents. Their usability scored highly across a range of metrics, with all surveyed researchers (n = 6) stating that they would be likely (or very likely) to use these tools in the future.

Conclusions: Based on this study, we would recommend Covidence and Rayyan to systematic reviewers looking for suitable and easy to use tools to support T&Ab screening within healthcare research. These two tools consistently demonstrated good alignment with user requirements. We acknowledge, however, the role of some of the other tools we considered in providing more specialist features that may be of great importance to many researchers.

--

Best wishes, Neil

Coordinator, HIFA Project on Evidence-Informed Policy and Practice (sponsorship opportunity)

http://www.hifa.org/projects/evidence-informed-policy-and-practice

Joint Coordinator, HIFA Project on Multilingualism (sponsorship opportunity)

http://www.hifa.org/projects/multilingualism

Let's build a future where people are no longer dying for lack of healthcare information - Join HIFA: www.hifa.org

HIFA profile: Neil Pakenham-Walsh is coordinator of the HIFA global health campaign (Healthcare Information For All - www.hifa.org ), a global community with more than 19,000 members in 177 countries, interacting on six global forums in four languages. Twitter: @hifa_org FB: facebook.com/HIFAdotORG neil@hifa.org