The Automated Systematic Search Deduplicator (ASySD): a rapid, open-source, interoperable tool to remove duplicate citations in biomedical systematic reviews

Background Researchers performing high-quality systematic reviews search across multiple databases to identify relevant evidence. However, the same publication is often retrieved from several databases. Identifying and removing such duplicates (deduplication) can be extremely time-consuming, but failure to remove these citations can lead to the wrongful inclusion of duplicate data. Many existing tools are not sensitive enough, lack interoperability with other tools, are not freely accessible, or are difficult to use without programming knowledge. Here, we report the performance of our Automated Systematic Search Deduplicator (ASySD), a novel tool to perform automated deduplication of systematic searches for biomedical reviews. Methods We evaluated ASySD's performance on 5 unseen biomedical systematic search datasets of various sizes (1845-79,880 citations). We compared the performance of ASySD with EndNote's automated deduplication option and with the Systematic Review Assistant Deduplication Module (SRA-DM). Results ASySD identified more duplicates than either SRA-DM or EndNote, with a sensitivity in different datasets of 0.95 to 0.99. The false-positive rate was comparable to human performance, with a specificity of > 0.99. The tool took less than 1 h to identify and remove duplicates within each dataset. Conclusions For duplicate removal in biomedical systematic reviews, ASySD is a highly sensitive, reliable, and timesaving tool. It is open source and freely available online as both an R package and a user-friendly web application.

Automated summary

This study presents the performance of ASySD, an automated tool for deduplicating systematic searches in biomedical reviews. ASySD outperforms both SRA-DM and EndNote in identifying and removing duplicates, with a sensitivity of 0.95 to 0.99 and a specificity of > 0.99. The tool is time-saving, reliable, and freely available online.