A randomized trial provided new evidence on the accuracy and efficiency of traditional vs. electronically annotated abstraction approaches in systematic reviews

Objectives: Data Abstraction Assistant (DAA) is a software for linking items abstracted into a data collection form for a systematic review to their locations in a study report. We conducted a randomized cross-over trial that compared DAA-facilitated single-data abstraction plus verification (DAA verification''), single data abstraction plus verification (regular verification''), and independent dual data abstraction plus adjudication (independent abstraction''). Study Design and Setting: This study is an online randomized cross-over trial with 26 pairs of data abstractors. Each pair abstracted data from six articles, two per approach. Outcomes were the proportion of errors and time taken. Results: Overall proportion of errors was 17% for DAA verification, 16% for regular verification, and 15% for independent abstraction. DAA verification was associated with higher odds of errors when compared with regular verification (adjusted odds ratio [OR] 5 1.08; 95% confidence interval [CI]: 0.99-1.17) or independent abstraction (adjusted OR 5 1.12; 95% CI: 1.03-1.22). For each article, DAA verification took 20 minutes (95% CI: 1-40) longer than regular verification, but 46 minutes (95% CI: 26 to 66) shorter than independent abstraction. Conclusion: Independent abstraction may only be necessary for complex data items. DAA provides an audit trail that is crucial for reproducible research. (C) 2019 The Authors. Published by Elsevier Inc.