Generalizability of deep learning models for dental image analysis

We assessed the generalizability of deep learning models and how to improve it. Our exemplary use-case was the detection of apical lesions on panoramic radiographs. We employed two datasets of panoramic radiographs from two centers, one in Germany (Charite, Berlin, n=650) and one in India (KGMU, Lucknow, n=650): First, U-Net type models were trained on images from Charite (n=500) and assessed on test sets from Charite and KGMU (each n=150). Second, the relevance of image characteristics was explored using pixel-value transformations, aligning the image characteristics in the datasets. Third, cross-center training effects on generalizability were evaluated by stepwise replacing Charite with KGMU images. Last, we assessed the impact of the dental status (presence of root-canal fillings or restorations). Models trained only on Charite images showed a (mean +/- SD) F1-score of 54.1 +/- 0.8% on Charite and 32.7 +/- 0.8% on KGMU data (p<0.001/t-test). Alignment of image data characteristics between the centers did not improve generalizability. However, by gradually increasing the fraction of KGMU images in the training set (from 0 to 100%) the F1-score on KGMU images improved (46.1 +/- 0.9%) at a moderate decrease on Charite images (50.9 +/- 0.9%, p<0.01). Model performance was good on KGMU images showing root-canal fillings and/or restorations, but much lower on KGMU images without root-canal fillings and/or restorations. Our deep learning models were not generalizable across centers. Cross-center training improved generalizability. Noteworthy, the dental status, but not image characteristics were relevant. Understanding the reasons behind limits in generalizability helps to mitigate generalizability problems.

Automated summary

The study evaluated the generalizability of deep learning models in detecting apical lesions on panoramic radiographs. Cross-center training improved the generalizability of the models. Furthermore, the dental status had a significant impact on model performance.