Inter-modality assessment of medial temporal lobe atrophy in a non-demented population: application of a visual rating scale template across radiologists with varying clinical experience

Objectives To assess inter-modality agreement and accuracy for medial temporal lobe atrophy (MTA) ratings across radiologists with varying clinical experience in a non-demented population. Methods Four raters (two junior radiologists and two senior neuroradiologists) rated MTA on CT and MRI scans using Scheltens' MTA scale. Ratings were compared to a consensus rating by two experienced neuroradiologists for estimation of true positive and negative rates (TPR and TNR) and over- and underestimation of MTA. Inter-modality agreement expressed as Cohen's kappa (dichotomized data), Cohen's kappa(w), and two-way mixed, single measures, consistency ICC (ordinal data) were determined. Adequate agreement was defined as kappa/kappa(w) >= 0.80 and ICC >= 0.80 (significance level at 95% CI >= 0.65). Results Forty-nine subjects (median age 72 years, 27% abnormal MTA) with cognitive impairment were included. Only junior radiologists achieved adequate agreement expressed as Cohen's kappa. All raters achieved adequate agreement expressed as Cohen's kappa(w) and ICC. True positive rates varied from 69 to 100% and TNR varied from 85 to 100%. No under- or overestimation of MTA was observed. Ratings did not differ between radiologists. Conclusion We conclude that radiologists with varying experience achieve adequate inter-modality agreement and similar accuracy when Scheltens' MTA scale is used to rate MTA on a non-demented population. However, TPR varied between radiologists which could be attributed to rating style differences.

Automated summary

The study found that radiologists with varying experience achieve satisfactory inter-modality agreement and similar accuracy when rating medial temporal lobe atrophy using Scheltens' MTA scale on a non-demented population. Junior radiologists achieved satisfactory agreement in Cohen's kappa, while all raters achieved satisfactory agreement in Cohen's kappa(w) and ICC. True positive rates and true negative rates varied between radiologists, but no underestimation or overestimation of MTA was observed.