4.4 Article

Automating risk of bias assessment in systematic reviews: a real-time mixed methods comparison of human researchers to a machine learning system

Related references

Note: Only part of the references are listed.
Review Mathematical & Computational Biology

Machine learning in systematic reviews: Comparing automated text clustering with Lingo3G and human researcher categorization in a rapid review

Ashley Elizabeth Muller et al.

Summary: The study evaluated the utility of an automated clustering method in categorizing studies, finding that automated clustering had higher precision and recall compared to manual categorization, and also saved 49% more time. The clustering algorithm was sensitive enough to group studies based on linguistic differences, corresponding to manual categories.

RESEARCH SYNTHESIS METHODS (2022)

Article Medicine, General & Internal

Text mining to support abstract screening for knowledge syntheses: a semi-automated workflow

Ba Pham et al.

Summary: A semi-automated workflow was developed for abstract screening in systematic reviews using text mining and machine learning methods, achieving high efficiency and accuracy. The workflow demonstrated good performance in practice, effectively reducing workload and improving review efficiency.

SYSTEMATIC REVIEWS (2021)

Article Biochemical Research Methods

Using neural networks to support high-quality evidence mapping

Thomas B. Rost et al.

Summary: The Living Evidence Map Project at the Norwegian Institute of Public Health provides an updated overview of research results and publications, with experts continuously monitoring, assessing, coding and summarizing new COVID-19 publications. Deep learning methods have been employed to aid manual classification, screening and preprocessing of the rapidly growing influx of new papers on the subject, showing acceptable performance in classifying publication topic and type.

BMC BIOINFORMATICS (2021)

Article Medicine, General & Internal

The views of health guideline developers on the use of automation in health evidence synthesis

Anneliese Arno et al.

Summary: The study analyzed the attitudes of guideline developers towards the use of automation in health evidence synthesis and found that compatibility with current values and practices was the primary concern, followed by relative advantage and observability. Participants expressed a desire for transparency in automation software methodology. Complexity and trialability were of less interest. The conclusions emphasized the importance of ensuring new technologies align with current values and maximizing transparency to address the concerns of guideline developers for wider use of machine learning and automation technologies in systematic reviews and guideline development.

SYSTEMATIC REVIEWS (2021)

Article Nursing

Agreement in Risk of Bias Assessment Between RobotReviewer and Human Reviewers: An Evaluation Study on Randomised Controlled Trials in Nursing-Related Cochrane Reviews

Julian Hirt et al.

Summary: RobotReviewer showed moderate agreement with human reviewers for randomization and allocation concealment in RCTs included in nursing-related Cochrane reviews, with adequate sensitivity for detecting low risk of selection bias. Human reviewers should supervise the semi-automated assessment process.

JOURNAL OF NURSING SCHOLARSHIP (2021)

Article Health Care Sciences & Services

Machine learning reduced workload with minimal risk of missing studies: development and evaluation of a randomized controlled trial classifier for Cochrane Reviews

James Thomas et al.

Summary: The study developed a machine learning classifier to reduce study identification workload in Cochrane for producing systematic reviews. The calibrated classifier showed high recall in evaluation, with older records more likely to be missed compared to newer records.

JOURNAL OF CLINICAL EPIDEMIOLOGY (2021)

Article Medical Informatics

Machine learning to help researchers evaluate biases in clinical trials: a prospective, randomized user study

Frank Soboczenski et al.

BMC MEDICAL INFORMATICS AND DECISION MAKING (2019)

Article Medicine, General & Internal

RoB 2: a revised tool for assessing risk of bias in randomised trials

Jonathan A. C. Sterne et al.

BMJ-BRITISH MEDICAL JOURNAL (2019)

Editorial Material Ecology

Software support for environmental evidence synthesis

Martin J. Westgate et al.

NATURE ECOLOGY & EVOLUTION (2018)

Article Computer Science, Information Systems

RobotReviewer: evaluation of a system for automatically assessing bias in clinical trials

Iain J. Marshall et al.

JOURNAL OF THE AMERICAN MEDICAL INFORMATICS ASSOCIATION (2016)

Review Medicine, General & Internal

Using text mining for study identification in systematic reviews: a systematic review of current approaches

Alison O'Mara-Eves et al.

SYSTEMATIC REVIEWS (2015)

Article Health Care Sciences & Services

Testing the Risk of Bias tool showed low reliability between individual reviewers and across consensus assessments of reviewer pairs

Lisa Hartling et al.

JOURNAL OF CLINICAL EPIDEMIOLOGY (2013)

Article Health Care Sciences & Services

GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables

Gordon Guyatt et al.

JOURNAL OF CLINICAL EPIDEMIOLOGY (2011)

Article Medicine, General & Internal

The Cochrane Collaboration's tool for assessing risk of bias in randomised trials

Julian P. T. Higgins et al.

BMJ-BRITISH MEDICAL JOURNAL (2011)

Article Medicine, General & Internal

How quickly do systematic reviews go out of date? A survival analysis

Kaveh G. Shojania et al.

ANNALS OF INTERNAL MEDICINE (2007)