4.6 Review

A Survey on Medical Explainable AI (XAI): Recent Progress, Explainability Approach, Human Interaction and Scoring System

Related references

Note: Only part of the references are listed.
Review Biochemical Research Methods

Impact of computational approaches in the fight against COVID-19: an AI guided review of 17 000 studies

Francesco Napolitano et al.

Summary: This article describes the widespread use of computational approaches in the COVID-19 pandemic and the efforts to collect, categorize, and prioritize relevant research articles. Machine learning methods were used to identify six application areas and conduct a systematic review of relevant studies. The significant contribution of computational applications during the ongoing pandemic will motivate further development and adoption.

BRIEFINGS IN BIOINFORMATICS (2022)

Article Computer Science, Artificial Intelligence

Unbox the black-box for the medical explainable AI via multi-modal and multi-centre data fusion: A mini-review, two showcases and beyond

Guang Yang et al.

Summary: XAI is an emerging research field in machine learning that aims to explain the decision-making process of AI systems. In healthcare, XAI is becoming increasingly important for improving the transparency and explainability of deep learning applications, although the lack of explainability in most AI systems may be a major barrier to successful implementation of AI tools in clinical practice.

INFORMATION FUSION (2022)

Article Computer Science, Artificial Intelligence

GraphXCOVID: Explainable deep graph diffusion pseudo-Labelling for identifying COVID-19 on chest X-rays

Angelica Aviles-Rivero et al.

Summary: Semi-supervised learning shows promise in COVID-19 diagnosis, outperforming supervised models with just a small fraction of labeled examples. Researchers introduce a graph-based deep semi-supervised framework that optimizes the relation between labeled and unlabeled data, and provide visualizations to assist radiologists in diagnosis.

PATTERN RECOGNITION (2022)

Article Computer Science, Information Systems

Improving the In-Hospital Mortality Prediction of Diabetes ICU Patients Using a Process Mining/Deep Learning Architecture

Julian Theis et al.

Summary: In this paper, a process mining/deep learning architecture is proposed to improve established severity scoring methods by incorporating the medical history of diabetes patients. The proposed method, Decay Replay Mining, combines medical and demographic information to predict the in-hospital mortality of diabetes ICU patients. Significant performance improvements are demonstrated compared to established methods.

IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS (2022)

Article Computer Science, Artificial Intelligence

Building and Interpreting Deep Similarity Models

Oliver Eberle et al.

Summary: This paper proposes a method to make similarities interpretable by decomposing deep similarity models and provides insights into complex similarity models. The method is applied to assess similarity between historical documents in digital humanities.

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE (2022)

Article Biochemistry & Molecular Biology

Application of interpretable machine learning for early prediction of prognosis in acute kidney injury

Chang Hu et al.

Summary: This study developed an algorithm using explainable artificial intelligence approaches for early prediction of mortality in intensive care unit patients with acute kidney injury. Machine learning models based on clinical features were developed and validated with great performance.

COMPUTATIONAL AND STRUCTURAL BIOTECHNOLOGY JOURNAL (2022)

Article Computer Science, Artificial Intelligence

Explainable artificial intelligence (XAI) in deep learning-based medical image analysis

Bas H. M. Van der Velden et al.

Summary: This survey examines the applications of explainable artificial intelligence (XAI) in deep learning-based medical image analysis. It introduces a framework for classifying deep learning-based medical image analysis methods based on XAI criteria. The survey also categorizes and investigates XAI techniques in medical image analysis according to the framework and anatomical location. The paper concludes by discussing future opportunities for XAI in medical image analysis.

MEDICAL IMAGE ANALYSIS (2022)

Article Chemistry, Multidisciplinary

Shape-Based Breast Lesion Classification Using Digital Tomosynthesis Images: The Role of Explainable Artificial Intelligence

Sardar Mehboob Hussain et al.

Summary: This study proposed an explainable deep-learning-driven multiclass shape-based classification framework for tomosynthesis breast lesion images. Experimental results validated the applicability of the framework and shed light on the black box nature of deep learning models, emphasizing the pros and cons of Grad-CAM and LIME methods.

APPLIED SCIENCES-BASEL (2022)

Article Multidisciplinary Sciences

Developing an ensemble machine learning model for early prediction of sepsis-associated acute kidney injury

Luming Zhang et al.

Summary: An ensemble model was developed to predict the onset of sepsis-associated acute kidney injury (S-AKI) early, and it showed good performance in multiple external datasets.

ISCIENCE (2022)

Article Medicine, Research & Experimental

Interpretable machine learning prediction of all-cause mortality

Wei Qiu et al.

Summary: This paper analyzes all-cause mortality using complex machine learning models and proposes a framework called IMPACT to explain mortality prediction. The study shows that the IMPACT models achieve higher accuracy than linear models and neural networks. By applying the IMPACT framework to the NHANES dataset, the authors identify overlooked risk factors and develop interpretable mortality risk scores. The study aims to bring explainable artificial intelligence to epidemiology.

COMMUNICATIONS MEDICINE (2022)

Article Urology & Nephrology

Incorporation of Urinary Neutrophil Gelatinase-Associated Lipocalin and Computed Tomography Quantification to Predict Acute Kidney Injury and In-Hospital Death in COVID-19 Patients

Li He et al.

Summary: This study developed and validated a prognostic model for AKI and in-hospital death in COVID-19 patients, incorporating the new tubular injury biomarker u-NGAL and AI-based chest CT analysis. The model showed high accuracy in predicting AKI and was validated internally.

KIDNEY DISEASES (2021)

Article Clinical Neurology

Mortality Prediction in Cerebral Hemorrhage Patients Using Machine Learning Algorithms in Intensive Care Units

Ximing Nie et al.

Summary: This study found that the random forest algorithm performed the best in predicting hospital mortality for cerebral hemorrhage patients in intensive care units, showing the highest specificity and accuracy.

FRONTIERS IN NEUROLOGY (2021)

Article Gastroenterology & Hepatology

Application of machine learning models for predicting acute kidney injury following donation after cardiac death liver transplantation

Zeng-Lei He et al.

Summary: The study found that a random forest model based on machine learning algorithms demonstrated stronger predictive power in forecasting AKI after donation after cardiac death LT compared to other models and conventional logistic regression model, with significantly higher AUC.

HEPATOBILIARY & PANCREATIC DISEASES INTERNATIONAL (2021)

Article Cardiac & Cardiovascular Systems

Left ventricular systolic dysfunction identification using artificial intelligence-augmented electrocardiogram in cardiac intensive care unit patients

Jacob C. Jentzer et al.

Summary: The AI-ECG demonstrated good accuracy in detecting LVSD, particularly in patients with ACS. Its performance was better in younger male patients and those without ACS.

INTERNATIONAL JOURNAL OF CARDIOLOGY (2021)

Article Urology & Nephrology

Systematic review and subgroup analysis of the incidence of acute kidney injury (AKI) in patients with COVID-19

Zhenjian Xu et al.

Summary: This systematic review evaluated the incidence of acute kidney injury (AKI) among hospitalized COVID-19 patients, finding a pooled estimated incidence of 10.0% and a pooled estimated proportion of patients needing continuous renal replacement therapy (CRRT) of 4%. Subgroup analysis suggested that age, disease severity, and ethnicity may be associated with the incidence of AKI in COVID-19 patients. Additionally, the study found that remdesivir treatment in COVID-19 patients was not associated with an increased risk of AKI.

BMC NEPHROLOGY (2021)

Article Urology & Nephrology

Convolutional Neural Network Model for Intensive Care Unit Acute Kidney Injury Prediction

Sidney Le et al.

Summary: This study successfully predicted the onset of acute kidney injury in patients using a machine learning algorithm and patient electronic health record data. The algorithm outperformed other prediction models and showed superior performance in predicting AKI 48 hours before onset, without reliance on serum creatinine measurements.

KIDNEY INTERNATIONAL REPORTS (2021)

Article Medical Informatics

Machine learning predicts mortality based on analysis of ventilation parameters of critically ill patients: multi-centre validation

Behrooz Mamandipoor et al.

Summary: In this study, the use of RNN model showed better predictive performance in mechanically ventilated patients, particularly in those with respiratory disorders. Furthermore, the inclusion of variables related to the function of other organs improved the predictive performance of the model.

BMC MEDICAL INFORMATICS AND DECISION MAKING (2021)

Article Computer Science, Information Systems

Responsible Artificial Intelligence in Healthcare: Predicting and Preventing Insurance Claim Denials for Economic and Social Wellbeing

Marina Johnson et al.

Summary: One out of seven health insurance claims is rejected in the US, causing hospitals to lose approximately $262 billion annually. Preventing claim denials improves profitability, accelerates the revenue cycle, and supports patients' wellbeing.

INFORMATION SYSTEMS FRONTIERS (2021)

Article Health Care Sciences & Services

Real-Time Clinical Decision Support Based on Recurrent Neural Networks for In-Hospital Acute Kidney Injury: External Validation and Model Interpretation

Kipyo Kim et al.

Summary: This study developed and externally validated a continuous prediction model for in-hospital AKI using recurrent neural networks. The models showed good performance in real-time assessment and personalized risk factor identification, providing support for clinical decisions.

JOURNAL OF MEDICAL INTERNET RESEARCH (2021)

Article Urology & Nephrology

A deep-learning model to continuously predict severe acute kidney injury based on urine output changes in critically ill patients

Francesca Alfieri et al.

Summary: This study aimed to compare the accuracy of a deep learning model based on urine output trends with logistic regression analysis in predicting the development of Acute Kidney Injury (AKI). The results showed that the deep learning model had a higher accuracy and was able to predict AKI episodes more than 12 hours in advance.

JOURNAL OF NEPHROLOGY (2021)

Article Computer Science, Artificial Intelligence

A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems

Sina Mohseni et al.

Summary: The demand for interpretable and accountable intelligent systems is increasing as artificial intelligence applications become more prevalent in everyday life. Researchers from various disciplines collaborate to define, design, and assess explainable AI systems. By categorizing XAI design goals and evaluation methods, this article aims to support different design objectives and evaluation methods in interdisciplinary XAI research.

ACM TRANSACTIONS ON INTERACTIVE INTELLIGENT SYSTEMS (2021)

Article Cell Biology

Predictors of Mortality in Critically Ill COVID-19 Patients Demanding High Oxygen Flow: A Thin Line between Inflammation, Cytokine Storm, and Coagulopathy

Viseslav Popadic et al.

Summary: In this study of 160 consecutive critically ill COVID-19 patients, IL-6, serum albumin, and D-dimer at ICU admission, along with CT severity score, were identified as independent predictors of mortality.

OXIDATIVE MEDICINE AND CELLULAR LONGEVITY (2021)

Review Engineering, Electrical & Electronic

A Review of Deep Learning in Medical Imaging: Imaging Traits, Technology Trends, Case Studies With Progress Highlights, and Future Promises

S. Kevin Zhou et al.

Summary: Deep learning has been applied in medical imaging tasks with remarkable success, however facing unique challenges. This survey article presents traits of medical imaging, clinical needs, technical challenges, and how emerging trends in DL are addressing these issues. Case studies include digital pathology and various imaging modalities in clinical practice.

PROCEEDINGS OF THE IEEE (2021)

Article Computer Science, Hardware & Architecture

The Ten Commandments of Ethical Medical AI

Heimo Mueller et al.

Summary: The proposed ten commandments serve as practical guidelines for those applying artificial intelligence, offering a concise checklist to a wide range of stakeholders.

COMPUTER (2021)

Article Biology

Interpretable heartbeat classification using local model-agnostic explanations on ECGs

Ines Neves et al.

Summary: This study focuses on an Explainable Artificial Intelligence (XAI) solution to improve the interpretability of heartbeat classification. By introducing a novel method that adds temporal dependency between time samples in time series data, the classification explanation becomes more reliable and trustworthy.

COMPUTERS IN BIOLOGY AND MEDICINE (2021)

Article Multidisciplinary Sciences

Explainable DCNN based chest X-ray image analysis and classification for COVID-19 pneumonia detection

Jie Hou et al.

Summary: This research developed a new diagnosis platform using a deep convolutional neural network (DCNN) to assist radiologists in distinguishing COVID-19 pneumonia from non-COVID-19 pneumonia based on chest X-ray analysis. The explainable method in the DCNN helps achieve higher prediction accuracy, with an average accuracy above 96%, and has the potential for large-scale rapid screening of COVID-9.

SCIENTIFIC REPORTS (2021)

Review Chemistry, Multidisciplinary

Current Challenges and Future Opportunities for XAI in Machine Learning-Based Clinical Decision Support Systems: A Systematic Review

Anna Markella Antoniadi et al.

Summary: Machine Learning and Artificial Intelligence have great potential in transforming medicine, but the lack of transparency in AI applications, especially in Clinical Decision Support Systems, can lead to issues of reliance. Explainable AI (XAI) provides rationale for users to understand AI outputs, but there is a distinct lack of XAI application and user studies in the context of CDSS. Further research is needed to explore the opportunities and challenges of implementing XAI in CDSS.

APPLIED SCIENCES-BASEL (2021)

Review Computer Science, Information Systems

A Systematic Review of Human-Computer Interaction and Explainable Artificial Intelligence in Healthcare With Artificial Intelligence Techniques

Mobeen Nazar et al.

Summary: Artificial intelligence (AI) and Human-Computer Interaction (HCI) have gained widespread acceptance in various fields, providing interactive intelligent systems for users. This research explores Explainable Artificial Intelligence (XAI) in healthcare, addressing the current applications and future challenges in the field.

IEEE ACCESS (2021)

Article Computer Science, Artificial Intelligence

A Survey on Neural Network Interpretability

Yu Zhang et al.

Summary: This study provides a comprehensive review of the interpretability of neural networks, clarifies the definition, and proposes a new taxonomy. The trust in deep learning systems is affected by the interpretability issue, which is also related to ethical problems. The interpretability of deep networks is a desired property for becoming powerful tools in other research fields.

IEEE TRANSACTIONS ON EMERGING TOPICS IN COMPUTATIONAL INTELLIGENCE (2021)

Review Engineering, Multidisciplinary

Multi-Disease Prediction Based on Deep Learning: A Survey

Shuxuan Xie et al.

Summary: The integration of AI and healthcare, particularly in the field of deep learning, has shown great potential for disease prediction and drug response. While there are still challenges, there is a promising future ahead for the application of deep learning in medical research and prediction.

CMES-COMPUTER MODELING IN ENGINEERING & SCIENCES (2021)

Review Health Care Sciences & Services

Deep learning-enabled medical computer vision

Andre Esteva et al.

Summary: The past decade has seen unprecedented progress in artificial intelligence, particularly in the field of medicine. Modern computer vision techniques powered by deep learning are revolutionizing medical applications, including imaging and video analysis. However, there are challenges to overcome for the real-world clinical deployment of these technologies.

NPJ DIGITAL MEDICINE (2021)

Article Medical Informatics

A new concordant partial AUC and partial c statistic for imbalanced data in the evaluation of machine learning algorithms

Andre M. Carrington et al.

BMC MEDICAL INFORMATICS AND DECISION MAKING (2020)

Review Computer Science, Hardware & Architecture

Techniques for Interpretable Machine Learning

Mengnan Du et al.

COMMUNICATIONS OF THE ACM (2020)

Article Medical Informatics

Use of AI-based tools for healthcare purposes: a survey study from consumers' perspectives

Pouyan Esmaeilzadeh

BMC MEDICAL INFORMATICS AND DECISION MAKING (2020)

Article Medical Informatics

Using machine learning methods to predict in-hospital mortality of sepsis patients in the ICU

Guilan Kong et al.

BMC MEDICAL INFORMATICS AND DECISION MAKING (2020)

Article Multidisciplinary Sciences

Explainable artificial intelligence model to predict acute critical illness from electronic health records

Simon Meyer Lauritsen et al.

NATURE COMMUNICATIONS (2020)

Article Computer Science, Artificial Intelligence

SurvLIME: A method for explaining machine learning survival models

Maxim S. Kovalev et al.

KNOWLEDGE-BASED SYSTEMS (2020)

Article Gastroenterology & Hepatology

Machine Learning Models of Acute Kidney Injury Prediction in Acute Pancreatitis Patients

Cheng Qu et al.

GASTROENTEROLOGY RESEARCH AND PRACTICE (2020)

Article Medical Informatics

Predicting mortality in critically ill patients with diabetes using machine learning and clinical notes

Jiancheng Ye et al.

BMC MEDICAL INFORMATICS AND DECISION MAKING (2020)

Proceedings Paper Computer Science, Artificial Intelligence

Doctor XAI An ontology-based approach to black-box sequential data classification explanations

Cecilia Panigutti et al.

FAT* '20: PROCEEDINGS OF THE 2020 CONFERENCE ON FAIRNESS, ACCOUNTABILITY, AND TRANSPARENCY (2020)

Review Computer Science, Information Systems

Explainable artificial intelligence models using real-world electronic health record data: a systematic scoping review

Seyedeh Neelufar Payrovnaziri et al.

JOURNAL OF THE AMERICAN MEDICAL INFORMATICS ASSOCIATION (2020)

Editorial Material Immunology

Antibiotic Therapy for Pseudomonas aeruginosa Bloodstream Infections: How Long Is Long Enough?

Valeria Fabre et al.

CLINICAL INFECTIOUS DISEASES (2019)

Article Multidisciplinary Sciences

An attention based deep learning model of clinical events in the intensive care unit

Deepak A. Kaji et al.

PLOS ONE (2019)

Article Multidisciplinary Sciences

Unmasking Clever Hans predictors and assessing what machines really learn

Sebastian Lapuschkin et al.

NATURE COMMUNICATIONS (2019)

Article Multidisciplinary Sciences

DeepSOFA: A Continuous Acuity Score for Critically III Patients using Clinically Interpretable Deep Learning

Benjamin Shickel et al.

SCIENTIFIC REPORTS (2019)

Article Multidisciplinary Sciences

A clinically applicable approach to continuous prediction of future acute kidney injury

Nenad Tomasev et al.

NATURE (2019)

Article Multidisciplinary Sciences

Definitions, methods, and applications in interpretable machine learning

W. James Murdoch et al.

PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA (2019)

Article Emergency Medicine

Development and Evaluation of a Machine Learning Model for the Early Identification of Patients at Risk for Sepsis

Ryan J. Delahanty et al.

ANNALS OF EMERGENCY MEDICINE (2019)

Article Neurosciences

Analyzing Neuroimaging Data Through Recurrent Deep Learning Models

Armin W. Thomas et al.

FRONTIERS IN NEUROSCIENCE (2019)

Article Computer Science, Interdisciplinary Applications

Mortality prediction in intensive care units (ICUs) using a deep rule-based fuzzy classifier

Raheleh Davoodi et al.

JOURNAL OF BIOMEDICAL INFORMATICS (2018)

Article Medicine, General & Internal

Catheter removal and outcomes of multidrug-resistant central-line-associated bloodstream infection

Jason P. Burnham et al.

MEDICINE (2018)

Article Medicine, General & Internal

Prediction of Acute Kidney Injury after Liver Transplantation: Machine Learning Approaches vs. Logistic Regression Model

Hyung-Chul Lee et al.

JOURNAL OF CLINICAL MEDICINE (2018)

Article Multidisciplinary Sciences

What is relevant in a text document?: An interpretable machine learning approach

Leila Arras et al.

PLOS ONE (2017)

Article Multidisciplinary Sciences

Combining Biomarkers with EMR Data to Identify Patients in Different Phases of Sepsis

Ishan Taneja et al.

SCIENTIFIC REPORTS (2017)

Article Biology

Learning representations for the early detection of sepsis with deep neural networks

Hye Jin Kam et al.

COMPUTERS IN BIOLOGY AND MEDICINE (2017)

Article Multidisciplinary Sciences

MIMIC-III, a freely accessible critical care database

Alistair E. W. Johnson et al.

SCIENTIFIC DATA (2016)

Article Statistics & Probability

INTERPRETABLE CLASSIFIERS USING RULES AND BAYESIAN ANALYSIS: BUILDING A BETTER STROKE PREDICTION MODEL

Benjamin Letham et al.

ANNALS OF APPLIED STATISTICS (2015)

Article Computer Science, Artificial Intelligence

Segmentation of anatomical structures in chest radiographs using supervised methods: a comparative study on a public database

B van Ginneken et al.

MEDICAL IMAGE ANALYSIS (2006)