Automated Coronary Artery Calcification Scoring in Non-Gated Chest CT: Agreement and Reliability

Objective: To determine the agreement and reliability of fully automated coronary artery calcium (CAC) scoring in a lung cancer screening population. Materials and Methods: 1793 low-dose chest CT scans were analyzed (non-contrast-enhanced, non-gated). To establish the reference standard for CAC, first automated calcium scoring was performed using a preliminary version of a method employing coronary calcium atlas and machine learning approach. Thereafter, each scan was inspected by one of four trained raters. When needed, the raters corrected initially automaticity-identified results. In addition, an independent observer subsequently inspected manually corrected results and discarded scans with gross segmentation errors. Subsequently, fully automatic coronary calcium scoring was performed. Agatston score, CAC volume and number of calcifications were computed. Agreement was determined by calculating proportion of agreement and examining Bland-Altman plots. Reliability was determined by calculating linearly weighted kappa (k) for Agatston strata and intraclass correlation coefficient (ICC) for continuous values. Results: 44 (2.5%) scans were excluded due to metal artifacts or gross segmentation errors. In the remaining 1749 scans, median Agatston score was 39.6 (P25-P75:0-345.9), median volume score was 60.4 mm(3) (P25-P75:0-361.4) and median number of calcifications was 2 (P25-P75:0-4) for the automated scores. The k demonstrated very good reliability (0.85) for Agatston risk categories between the automated and reference scores. The Bland-Altman plots showed underestimation of calcium score values by automated quantification. Median difference was 2.5 (p25-p75:0.0-53.2) for Agatston score, 7.6 (p25-p75:0.0-94.4) for CAC volume and 1 (p25-p75:0-5) for number of calcifications. The ICC was very good for Agatston score (0.90), very good for calcium volume (0.88) and good for number of calcifications (0.64). Discussion: Fully automated coronary calcium scoring in a lung cancer screening setting is feasible with acceptable reliability and agreement despite an underestimation of the amount of calcium when compared to reference scores.