Automatic correction of performance drift under acquisition shift in medical image classification

Image-based prediction models for disease detection are sensitive to changes in data acquisition such as the replacement of scanner hardware or updates to the image processing software. The resulting differences in image characteristics may lead to drifts in clinically relevant performance metrics which could cause harm in clinical decision making, even for models that generalise in terms of area under the receiver-operating characteristic curve. We propose Unsupervised Prediction Alignment, a generic automatic recalibration method that requires no ground truth annotations and only limited amounts of unlabelled example images from the shifted data distribution. We illustrate the effectiveness of the proposed method to detect and correct performance drift in mammography-based breast cancer screening and on publicly available histopathology data. We show that the proposed method can preserve the expected performance in terms of sensitivity/specificity under various realistic scenarios of image acquisition shift, thus offering an important safeguard for clinical deployment. Automatic correction of performance drift caused by changes in image acquisition is key for safe AI deployment. Here, the authors present a solution that restores the expected clinical performance of image classification systems in breast screening and histopathology.

Automated summary

Image-based prediction models are sensitive to changes in data acquisition. We propose an unsupervised method that can detect and correct performance drift, offering an important safeguard for clinical deployment.