4.6 Article

Natural language processing was effective in assisting rapid title and abstract screening when updating systematic reviews

Journal

JOURNAL OF CLINICAL EPIDEMIOLOGY
Volume 133, Issue -, Pages 121-129

Publisher

ELSEVIER SCIENCE INC
DOI: 10.1016/j.jclinepi.2021.01.0100895-4356/

Keywords

Machine learning; Natural language processing; Systematic review update; Title and abstract screening; BERT; LightGBM

Funding

  1. National Key R&D Program of China [2019YFC1709804, 2017YFC1700406]
  2. National Natural Science Foundation of China [71904134]
  3. Sichuan Youth Science and Technology Innovation Research Team [2020JDTD0015]
  4. China Postdoctoral Science Foundation [2019M653444]
  5. 1 . 3 . 5 project for disciplines of excellence, West China Hospital, Sichuan University [ZYYC08003]

Ask authors/readers for more resources

This study examines the effectiveness of NLP technology in assisting rapid literature screening when updating systematic reviews. Using a LightGBM model, the study achieved high sensitivity and specificity, indicating that NLP technology can reduce reviewers' workload and improve screening efficiency.
Background and Objective: To examine whether the use of natural language processing (NLP) technology is effective in assisting rapid title and abstract screening when updating a systematic review. Study Design: Using the searched literature from a published systematic review, we trained and tested an NLP model that enables rapid title and abstract screening when updating a systematic review. The model was a light gradient boosting machine (LightGBM), an ensemble learning classifier which integrates four pretrained Bidirectional Encoder Representations from Transformers (BERT) models. We divided the searched citations into two sets (ie, training and test sets). The model was trained using the training set and assessed for screening performance using the test set. The searched citations, whose eligibility was determined by two independent reviewers, were treated as the reference standard. Results: The test set included 947 citations; our model included 340 citations, excluded 607 citations, and achieved 96% sensitivity, and 78% specificity. If the classifier assessment in the case study was accepted, reviewers would lose 8 of 180 eligible citations (4%), none of which were ultimately included in the systematic review after full-text consideration, while decreasing the workload by 64.1%. Conclusion: NLP technology using the ensemble learning method may effectively assist in rapid literature screening when updating systematic reviews. (c) 2021 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license ( http://creativecommons.org/licenses/by-nc-nd/4.0/ )

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.6
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

No Data Available
No Data Available