Journal
JOURNAL OF BIOMEDICAL INFORMATICS
Volume 133, Issue -, Pages -Publisher
ACADEMIC PRESS INC ELSEVIER SCIENCE
DOI: 10.1016/j.jbi.2022.104161
Keywords
Hierarchical label-wise attention; Transformers; Explainability; ICD coding; MIMIC-III
Funding
- Australian government
- Commonwealth Industrial and Scientific Research Organisation (CSIRO)
Ask authors/readers for more resources
In this study, a hierarchical label-wise attention Transformer model (HiLAT) is proposed for the explainable prediction of ICD codes from clinical documents. The model utilizes a pretrained Transformer model and a hierarchical label-wise attention mechanism to predict the assignment of specific ICD codes to clinical documents. Experimental results show that HiLAT + ClinicalplusXLNet outperforms previous state-of-the-art models for predicting ICD-9 codes, and attention weight visualizations serve as a potential tool for validating ICD code predictions.
International Classification of Diseases (ICD) coding plays an important role in systematically classifying morbidity and mortality data. In this study, we propose a hierarchical label-wise attention Transformer model (HiLAT) for the explainable prediction of ICD codes from clinical documents. HiLAT firstly fine-tunes a pretrained Transformer model to represent the tokens of clinical documents. We subsequently employ a two-level hierarchical label-wise attention mechanism that creates label-specific document representations. These representations are in turn used by a feed-forward neural network to predict whether a specific ICD code is assigned to the input clinical document of interest. We evaluate HiLAT using hospital discharge summaries and their corresponding ICD-9 codes from the MIMIC-III database. To investigate the performance of different types of Transformer models, we develop ClinicalplusXLNet, which conducts continual pretraining from XLNet-Base using all the MIMIC-III clinical notes. The experiment results show that the F1 scores of the HiLAT + ClinicalplusXLNet outperform the previous state-of-the-art models for the top-50 most frequent ICD-9 codes from MIMIC-III. Visualisations of attention weights present a potential explainability tool for checking the face validity of ICD code predictions.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available