4.7 Review

Methods for Clinical Evaluation of Artificial Intelligence Algorithms for Medical Diagnosis

相关参考文献

注意:仅列出部分参考文献,下载原文获取全部文献信息。
Article Radiology, Nuclear Medicine & Medical Imaging

Content-based Image Retrieval by Using Deep Learning for Interstitial Lung Disease Diagnosis with Chest CT

Jooae Choe et al.

Summary: A content-based image retrieval system using deep learning can improve the diagnostic accuracy and interreader agreement of interstitial lung disease in readers with different levels of experience.

RADIOLOGY (2022)

Article Radiology, Nuclear Medicine & Medical Imaging

Reduced-Dose Deep Learning Reconstruction for Abdominal CT of Liver Metastases

Corey T. Jensen et al.

Summary: This study compared reduced-dose deep learning image reconstruction with standard-dose filtered back projection contrast-enhanced abdominal CT in evaluating liver metastases and image quality. The results showed that deep learning image reconstruction improved CT image quality while reducing radiation dose, with superior performance in detecting liver lesions larger than 0.5 cm.

RADIOLOGY (2022)

Review Gastroenterology & Hepatology

Performance of artificial intelligence in colonoscopy for adenoma and polyp detection: a systematic review and meta-analysis

Cesare Hassan et al.

Summary: The meta-analysis found that incorporating artificial intelligence as an aid for detecting colorectal neoplasia significantly increases the detection rate of colorectal neoplasia, independently of the main adenoma characteristics.

GASTROINTESTINAL ENDOSCOPY (2021)

Article Radiology, Nuclear Medicine & Medical Imaging

Ultra-low-dose chest CT imaging of COVID-19 patients using a deep residual neural network

Isaac Shiri et al.

Summary: The study aimed to design a ultra-low-dose CT examination protocol for clinical diagnosis of COVID-19 patients using a deep learning approach. By utilizing a residual convolutional neural network, the study demonstrated the capability of predicting full-dose CT images with acceptable quality while substantially reducing radiation dose.

EUROPEAN RADIOLOGY (2021)

Article Radiology, Nuclear Medicine & Medical Imaging

Artificial Intelligence Applied to Breast MRI for Improved Diagnosis

Yulei Jiang et al.

Summary: In dynamic contrast material-enhanced (DCE) breast MRI, the use of an artificial intelligence system improves radiologists' performance in differentiating benign and malignant lesions, with improvements in average sensitivity and AUC score.

RADIOLOGY (2021)

Article Radiology, Nuclear Medicine & Medical Imaging

Deep learning-assisted differentiation of pathologically proven atypical and typical hepatocellular carcinoma (HCC) versus non-HCC on contrast-enhanced MRI of the liver

Paula M. Oestmann et al.

Summary: This study successfully trained a deep learning model to differentiate between pathologically confirmed HCC and non-HCC lesions on MRI, with good overall accuracy but lower accuracy for lesions with more atypical imaging features.

EUROPEAN RADIOLOGY (2021)

Article Radiology, Nuclear Medicine & Medical Imaging

To buy or not to buy-evaluating commercial AI solutions in radiology (the ECLAIR guidelines)

Patrick Omoumi et al.

Summary: In recent years, artificial intelligence has made significant progress in medical imaging, leading to the availability of numerous commercial AI solutions that require careful assessment before purchase. The ECLAIR guidelines proposed by authors from academia and industry offer a practical framework to help stakeholders evaluate commercial AI solutions in radiology, addressing factors such as relevance, performance, validation, usability, integration, regulatory aspects, and financial considerations.

EUROPEAN RADIOLOGY (2021)

Letter Biochemistry & Molecular Biology

A quality assessment tool for artificial intelligence-centered diagnostic test accuracy studies: QUADAS-AI

Viknesh Sounderajah et al.

NATURE MEDICINE (2021)

Review Surgery

Artificial Intelligence-Aided Colonoscopy for Polyp Detection: A Systematic Review and Meta-Analysis of Randomized Clinical Trials

Yuanchuan Zhang et al.

Summary: The study showed that AI-aided colonoscopy significantly improved the polyp detection rate and adenoma detection rate, especially for smaller polyps. However, further improvement is needed for the shape and pathology recognition of the AI technique.

JOURNAL OF LAPAROENDOSCOPIC & ADVANCED SURGICAL TECHNIQUES (2021)

Article Radiology, Nuclear Medicine & Medical Imaging

Artificial Intelligence Algorithm Improves Radiologist Performance in Skeletal Age Assessment: A Prospective Multicenter Randomized Controlled Trial

David K. Eng et al.

Summary: This study aimed to compare the accuracy and interpretation time of skeletal age assessment on hand radiograph examinations with and without the use of an AI algorithm as a diagnostic aid. Results showed that using the AI algorithm led to lower mean absolute difference, lower proportions of absolute differences exceeding 12 months and 24 months, and shorter interpretation time.

RADIOLOGY (2021)

Article Radiology, Nuclear Medicine & Medical Imaging

MRI-based Synthetic CT in the Detection of Structural Lesions in Patients with Suspected Sacroiliitis: Comparison with MRI

Lennart B. O. Jans et al.

Summary: The study findings suggest that MRI-based synthetic CT has superior diagnostic performance in detecting structural lesions in the sacroiliac joints compared to traditional T1-weighted MRI, showing higher accuracy in the detection of erosions, sclerosis, and ankylosis.

RADIOLOGY (2021)

Article Radiology, Nuclear Medicine & Medical Imaging

Computer-aided Detection of Subsolid Nodules at Chest CT: Improved Performance with Deep Learning-based CT Section Thickness Reduction

Sohee Park et al.

Summary: This study evaluated the impact of different CT section thicknesses on CAD performance in the detection of subsolid nodules. It was found that 1-mm section thickness CT achieved better detection results, especially for nonsolid nodules. Additionally, the use of a super-resolution algorithm improved CAD sensitivity at 3- and 5-mm section thickness CT.

RADIOLOGY (2021)

Article Radiology, Nuclear Medicine & Medical Imaging

Evaluation and Real-World Performance Monitoring of Artificial Intelligence Models in Clinical Practice: Try It, Buy It, Check It

Bibb Allen et al.

Summary: This article discusses the accelerating pace of regulatory clearance for AI algorithms in radiology and the potential issues with using these algorithms beyond their training institutions. It emphasizes that regulatory clearance alone may not be enough to ensure safety and efficacy in all radiological practices, and reviews strategies for evaluating and monitoring the performance of AI models.

JOURNAL OF THE AMERICAN COLLEGE OF RADIOLOGY (2021)

Article Clinical Neurology

Real-World Experience with Artificial Intelligence-Based Triage in Transferred Large Vessel Occlusion Stroke Patients

Jacob R. Morey et al.

Summary: The implementation of Viz LVO is associated with earlier and more consistent neuroendovascular team notification times, potentially leading to improved clinical outcomes for LVO stroke patients.

CEREBROVASCULAR DISEASES (2021)

Review Gastroenterology & Hepatology

Artificial intelligence (AI) real-time detection vs. routine colonoscopy for colorectal neoplasia: a meta-analysis and trial sequential analysis

Smit S. Deliwala et al.

Summary: This systematic review and meta-analysis evaluated the efficacy of AI-assisted colonoscopies against routine colonoscopies, finding that AI outperformed RC in detecting adenomas and polyps, but was less effective in detecting pedunculated polyps. The findings suggest that AI-assisted colonoscopy could bridge critical gaps in CRC identification.

INTERNATIONAL JOURNAL OF COLORECTAL DISEASE (2021)

Review Health Care Sciences & Services

Study designs for comparative diagnostic test accuracy: A methodological review and classification scheme

Bada Yang et al.

Summary: The study aimed to identify and classify comparative diagnostic test accuracy study designs, as well as describe study design labels used by authors. Results showed 46 unique combinations of study design features based on participant flow elements, classified into five design categories. This classification scheme can assist systematic review authors in defining study eligibility criteria and communicating evidence strength.

JOURNAL OF CLINICAL EPIDEMIOLOGY (2021)

Review Gastroenterology & Hepatology

Computer-aided detection versus advanced imaging for detection of colorectal neoplasia: a systematic review and network meta-analysis

Marco Spadaccini et al.

Summary: Based on systematic review and network meta-analysis, it was found that CADe has a higher detection rate for colorectal neoplasia compared to other techniques (such as chromoendoscopy or tools that increase mucosal visualization), supporting greater incorporation of CADe strategies into community endoscopy services.

LANCET GASTROENTEROLOGY & HEPATOLOGY (2021)

Editorial Material Biochemistry & Molecular Biology

How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals

Eric Wu et al.

Summary: A comprehensive overview of medical AI devices approved by the US Food and Drug Administration sheds light on limitations of the evaluation process that may mask vulnerabilities of devices when deployed on patients.

NATURE MEDICINE (2021)

Review Health Care Sciences & Services

Clinical impact and quality of randomized controlled trials involving interventions evaluating artificial intelligence prediction tools: a systematic review

Qian Zhou et al.

Summary: The study found that the evidence of the impact of TS, ML, and DL tools in clinical practice was limited, with DL applications not yet fully spread in medicine. In the future, DL may integrate more complex clinical problems than ML and TS tools. Rigorous studies are required before the clinical application of these tools.

NPJ DIGITAL MEDICINE (2021)

Article Radiology, Nuclear Medicine & Medical Imaging

Added Value of Deep Learning-based Detection System for Multiple Major Findings on Chest Radiographs: A Randomized Crossover Study

Jinkyeong Sung et al.

Summary: The study demonstrated that a deep learning-based detection system can enhance observer performance in detecting and localizing major abnormal findings on chest radiographs and reduce reading time.

RADIOLOGY (2021)

Article Clinical Neurology

Diagnostic Accuracy and Failure Mode Analysis of a Deep Learning Algorithm for the Detection of Cervical Spine Fractures

A. F. Voter et al.

Summary: This study evaluated the performance of an artificial intelligence decision support system, Aidoc, for the detection of cervical spinal fractures, revealing poor diagnostic accuracy. Concerns were raised about the generalizability, utility, and rapid deployment of similar algorithms that have not undergone external validation. Further rigorous evaluations are needed before widespread implementation of these tools.

AMERICAN JOURNAL OF NEURORADIOLOGY (2021)

Article Computer Science, Information Systems

The need to separate the wheat from the chaff in medical informatics Introducing a comprehensive checklist for the (self)-assessment of medical AI studies

Federico Cabitza et al.

Summary: The editorial proposes a practical checklist to help authors self-assess the quality of their contributions and aid reviewers in distinguishing high-quality medical ML studies from the mere application of ML techniques to medical data.

INTERNATIONAL JOURNAL OF MEDICAL INFORMATICS (2021)

Review Radiology, Nuclear Medicine & Medical Imaging

A review of medical image data augmentation techniques for deep learning applications

Phillip Chlap et al.

Summary: Data augmentation has become a popular method for training deep learning models in the field of radiology and radiotherapy, where limited medical image datasets are often encountered. By generating additional training data, data augmentation can improve model performance and has been widely used in state-of-the-art deep learning models.

JOURNAL OF MEDICAL IMAGING AND RADIATION ONCOLOGY (2021)

Article Radiology, Nuclear Medicine & Medical Imaging

Deep Learning for Detection of Pulmonary Metastasis on Chest Radiographs

Eui Jin Hwang et al.

Summary: This study evaluated the diagnostic yield of a deep learning-based CAD system for newly visible lung metastasis on chest radiographs in cancer patients, showing that the CAD system improved diagnostic yield while maintaining a similar false-referral rate.

RADIOLOGY (2021)

Article Medicine, General & Internal

Protocol for development of a reporting guideline (TRIPOD-AI) and risk of bias tool (PROBAST-AI) for diagnostic and prognostic prediction model studies based on artificial intelligence

Gary S. Collins et al.

Summary: This paper outlines the development process of TRIPOD-AI and PROBAST-AI, including systematic reviews, Delphi process, virtual consensus meetings, and tool development, in five stages. The aim is to provide reporting guidelines and a standardized tool for the critical appraisal of machine learning based prediction model studies.

BMJ OPEN (2021)

Article Medicine, General & Internal

Developing a reporting guideline for artificial intelligence-centred diagnostic test accuracy studies: the STARD-AI protocol

Viknesh Sounderajah et al.

Summary: The development of the STARD-AI checklist consists of six stages: project organization, item generation, modified Delphi consensus methodology, drafting the checklist, piloting phase, and dissemination and implementation strategy. The anticipated dissemination is set to take place in Q3 of 2021.

BMJ OPEN (2021)

Article Medicine, General & Internal

A Data Set and Deep Learning Algorithm for the Detection of Masses and Architectural Distortions in Digital Breast Tomosynthesis Images

Mateusz Buda et al.

Summary: This study aims to provide a large-scale DBT image dataset for the development and evaluation of artificial intelligence algorithms for breast cancer screening, while also developing a baseline deep learning model and testing it using the dataset.

JAMA NETWORK OPEN (2021)

Article Computer Science, Artificial Intelligence

Toward Generalizability in the Deployment of Artificial Intelligence in Radiology: Role of Computation Stress Testing to Overcome Underspecification

Thomas Eche et al.

Summary: The deployment of artificial intelligence in medical imaging poses challenges due to overfitting and underspecification. Generalizability is hindered by the inability of AI models to adapt to heterogeneous populations and imaging protocols. Stress testing is a key strategy to ensure broad generalizability of AI models in radiology.

RADIOLOGY-ARTIFICIAL INTELLIGENCE (2021)

Article Computer Science, Artificial Intelligence

Integrating Al Algorithms into the Clinical Workflow

Krishna Juluru et al.

Summary: The study describes generalizable components for integrating AI systems into clinical workflows, including deployment, usage assessment, and real-time performance monitoring. Results showed that the AI system successfully processed a large number of examinations and allowed radiologists to make real-time corrections to the results. Surveys indicated that the majority of users were satisfied with the integration of the AI system into the clinical workflow.

RADIOLOGY-ARTIFICIAL INTELLIGENCE (2021)

Article Computer Science, Artificial Intelligence

Training Strategies for Radiology Deep Learning Models in Data-limited Scenarios

Sema Candemir et al.

Summary: Data-driven approaches have great potential in shaping future practices in radiology, but face challenges such as patient privacy concerns, tedious annotation processes, and limited expert resources. This review discusses model training strategies in scenarios with limited data, insufficiently labeled data, and/or limited expert resources, including enlarging data samples, decreasing manual labeling burdens, adjusting network architectures, and leveraging pretrained models.

RADIOLOGY-ARTIFICIAL INTELLIGENCE (2021)

Review Radiology, Nuclear Medicine & Medical Imaging

Review of Statistical Methods for Evaluating the Performance of Survival or Other Time-to-Event Prediction Models (from Conventional to Deep Learning Approaches)

Seo Young Park et al.

Summary: The recent introduction of various high-dimensional modeling methods has increased the diversity of modeling approaches for survival prediction, but confusion may arise due to the novelty of these approaches. This article aims to provide intuitive, conceptual, and practical explanations of statistical methods for evaluating the performance of survival prediction models.

KOREAN JOURNAL OF RADIOLOGY (2021)

Review Radiology, Nuclear Medicine & Medical Imaging

Key Principles of Clinical Validation, Device Approval, and Insurance Coverage Decisions of Artificial Intelligence

Seong Ho Park et al.

Summary: Clinical validation of AI should involve proper external testing and assessing its impact on patient outcomes. It is up to medical professionals to determine if approved AI algorithms are beneficial for real-world patient care, and insurance coverage decisions usually require proof that the use of AI has improved patient outcomes.

KOREAN JOURNAL OF RADIOLOGY (2021)

Article Radiology, Nuclear Medicine & Medical Imaging

Imaging Predictors of Survival in Patients with Single Small Hepatocellular Carcinoma Treated with Transarterial Chemoembolization

Chan Park et al.

Summary: The study indicates that pre-TACE CT or MR imaging findings can predict survival outcomes in patients with small HCC undergoing TACE treatment, aiding in prognosis, identification, and selection of suitable candidates for TACE.

KOREAN JOURNAL OF RADIOLOGY (2021)

Article Radiology, Nuclear Medicine & Medical Imaging

Deep learning-based MR-to-CT synthesis: The influence of varying gradient echo-based MR images as input channels

Mateusz C. Florkow et al.

MAGNETIC RESONANCE IN MEDICINE (2020)

Article Radiology, Nuclear Medicine & Medical Imaging

Integrating artificial intelligence into the clinical practice of radiology: challenges and recommendations

Michael P. Recht et al.

EUROPEAN RADIOLOGY (2020)

Article Ophthalmology

A Clinician's Guide to Artificial Intelligence: How to Critically Appraise Machine Learning Studies

Livia Faes et al.

TRANSLATIONAL VISION SCIENCE & TECHNOLOGY (2020)

Article Computer Science, Information Systems

MINIMAR (MINimum Information for Medical Al Reporting): Developing reporting standards for artificial intelligence in health care

Tina Hernandez-Boussard et al.

JOURNAL OF THE AMERICAN MEDICAL INFORMATICS ASSOCIATION (2020)

Editorial Material Biochemistry & Molecular Biology

Minimum information about clinical artificial intelligence modeling: the MI-CLAIM checklist

Beau Norgeot et al.

NATURE MEDICINE (2020)

Article Biochemistry & Molecular Biology

Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension

Samantha Cruz Rivera et al.

NATURE MEDICINE (2020)

Article Medicine, General & Internal

Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI Extension

Samantha Cruz Rivera et al.

BMJ-BRITISH MEDICAL JOURNAL (2020)

Article Radiology, Nuclear Medicine & Medical Imaging

Evaluating Artificial Intelligence Systems to Guide Purchasing Decisions

Ross W. Filice et al.

JOURNAL OF THE AMERICAN COLLEGE OF RADIOLOGY (2020)

Article Health Care Sciences & Services

The state of artificial intelligence-based FDA-approved medical devices and algorithms: an online database

Stan Benjamens et al.

NPJ DIGITAL MEDICINE (2020)

Review Radiology, Nuclear Medicine & Medical Imaging

Basics of Deep Learning: A Radiologist's Guide to Understanding Published Radiology Articles on Deep Learning

Synho Do et al.

KOREAN JOURNAL OF RADIOLOGY (2020)

Editorial Material Multidisciplinary Sciences

Regulation of predictive analytics in medicine

Ravi B. Parikh et al.

SCIENCE (2019)

Article Medicine, General & Internal

Key challenges for delivering clinical impact with artificial intelligence

Christopher J. Kelly et al.

BMC MEDICINE (2019)

Article Medicine, General & Internal

Calibration: the Achilles heel of predictive analytics

Ben van Calster et al.

BMC MEDICINE (2019)

Article Radiology, Nuclear Medicine & Medical Imaging

Deep Learning Algorithm for Reducing CT Slice Thickness: Effect on Reproducibility of Radiomic Features in Lung Cancer

Sohee Park et al.

KOREAN JOURNAL OF RADIOLOGY (2019)

Article Biochemistry & Molecular Biology

Automated deep-neural-network surveillance of cranial images for acute neurologic events

Joseph J. Titano et al.

NATURE MEDICINE (2018)

Article Radiology, Nuclear Medicine & Medical Imaging

Radiomic MRI Phenotyping of Glioblastoma: Improving Survival Preciction

Sohi Bae et al.

RADIOLOGY (2018)

Editorial Material Medicine, General & Internal

Using Free-Response Receiver Operating Characteristic Curves to Assess the Accuracy of Machine Diagnosis of Cancer

Chaya S. Moskowitz

JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION (2017)

Review Oncology

Radiomics: the bridge between medical imaging and personalized medicine

Philippe Lambin et al.

NATURE REVIEWS CLINICAL ONCOLOGY (2017)

Article Obstetrics & Gynecology

Central Fetal Monitoring With and Without Computer Analysis A Randomized Controlled Trial

Ines Nunes et al.

OBSTETRICS AND GYNECOLOGY (2017)

Article Health Care Sciences & Services

Time-dependent ROC curve analysis in medical research: current methods and applications

Adina Najwa Kamarudin et al.

BMC MEDICAL RESEARCH METHODOLOGY (2017)

Article Medical Laboratory Technology

On determining the most appropriate test cut-off value: the case of tests with continuous results

Farrokh Habibzadeh et al.

BIOCHEMIA MEDICA (2016)

Editorial Material Medicine, General & Internal

The Propensity Score

Jason S. Haukoos et al.

JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION (2015)

Editorial Material Medicine, General & Internal

Randomised controlled trials: understanding confounding

Philip Sedgwick

BMJ-BRITISH MEDICAL JOURNAL (2015)

Article Radiology, Nuclear Medicine & Medical Imaging

Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool

Abdel Aziz Taha et al.

BMC MEDICAL IMAGING (2015)

Editorial Material Medicine, General & Internal

Randomised controlled trials: understanding confounding

Philip Sedgwick

BMJ-BRITISH MEDICAL JOURNAL (2015)

Article Radiology, Nuclear Medicine & Medical Imaging

ROC, LROC, FROC, AFROC: An Alphabet Soup

Xin He et al.

Journal of the American College of Radiology (2009)

Article Radiology, Nuclear Medicine & Medical Imaging

Receiver operating characteristic curves and their use in radiology

NA Obuchowski

RADIOLOGY (2003)