Natural language processing was effective in assisting rapid title and abstract screening when updating systematic reviews

Background and Objective: To examine whether the use of natural language processing (NLP) technology is effective in assisting rapid title and abstract screening when updating a systematic review. Study Design: Using the searched literature from a published systematic review, we trained and tested an NLP model that enables rapid title and abstract screening when updating a systematic review. The model was a light gradient boosting machine (LightGBM), an ensemble learning classifier which integrates four pretrained Bidirectional Encoder Representations from Transformers (BERT) models. We divided the searched citations into two sets (ie, training and test sets). The model was trained using the training set and assessed for screening performance using the test set. The searched citations, whose eligibility was determined by two independent reviewers, were treated as the reference standard. Results: The test set included 947 citations; our model included 340 citations, excluded 607 citations, and achieved 96% sensitivity, and 78% specificity. If the classifier assessment in the case study was accepted, reviewers would lose 8 of 180 eligible citations (4%), none of which were ultimately included in the systematic review after full-text consideration, while decreasing the workload by 64.1%. Conclusion: NLP technology using the ensemble learning method may effectively assist in rapid literature screening when updating systematic reviews. (c) 2021 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license ( http://creativecommons.org/licenses/by-nc-nd/4.0/ )

Automated summary

This study examines the effectiveness of NLP technology in assisting rapid literature screening when updating systematic reviews. Using a LightGBM model, the study achieved high sensitivity and specificity, indicating that NLP technology can reduce reviewers' workload and improve screening efficiency.