Journal
PLOS ONE
Volume 17, Issue 1, Pages -Publisher
PUBLIC LIBRARY SCIENCE
DOI: 10.1371/journal.pone.0262838
Keywords
-
Categories
Funding
- Intramural Research Program (IRP) of the National Library of Medicine (NLM)
- National Institutes of Health (NIH)
Ask authors/readers for more resources
In this study, the effect of model calibration on the performance of medical image classification tasks was systematically analyzed. The results show that calibration can significantly improve performance at the default classification threshold, but the differences are not significant at the PR-guided threshold. This observation holds for different image modalities and degrees of class imbalance.
In medical image classification tasks, it is common to find that the number of normal samples far exceeds the number of abnormal samples. In such class-imbalanced situations, reliable training of deep neural networks continues to be a major challenge, therefore biasing the predicted class probabilities toward the majority class. Calibration has been proposed to alleviate some of these effects. However, there is insufficient analysis explaining whether and when calibrating a model would be beneficial. In this study, we perform a systematic analysis of the effect of model calibration on its performance on two medical image modalities, namely, chest X-rays and fundus images, using various deep learning classifier backbones. For this, we study the following variations: (i) the degree of imbalances in the dataset used for training; (ii) calibration methods; and (iii) two classification thresholds, namely, default threshold of 0.5, and optimal threshold from precision-recall (PR) curves. Our results indicate that at the default classification threshold of 0.5, the performance achieved through calibration is significantly superior (p < 0.05) to using uncalibrated probabilities. However, at the PR-guided threshold, these gains are not significantly different (p > 0.05). This observation holds for both image modalities and at varying degrees of imbalance. The code is available at https://github.com/sivaramakrishnan-rajaraman/Model_calibration.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available