Evaluation of Reader Variability in the Interpretation of Follow-up CT Scans at Lung Cancer Screening

Purpose: To measure reader agreement in determining whether lung nodules detected at baseline screening computed tomography (CT) had changed at subsequent screening examinations and to evaluate the variability in recommendations for further follow-up. Materials and Methods: All subjects were enrolled in the National Lung Screening Trial (NLST), and each participant consented to the use of their de-identified images for research purposes. The authors randomly selected 100 cases of nodules measuring at least 4.0 mm at 1-year screening CT that were considered by the original screening CT reader to be present on baseline CT scans; nodules considered by the original reader to have changed were oversampled. Selected images from each case showing the entire nodule at both examinations were preloaded on a picture archiving and communication system workstation. Nine radiologists served as readers, and they evaluated whether the nodule was present at baseline and recorded the bidimensional measurements and nodule characteristics at each examination, presence or absence of change, results of screening CT, and follow-up recommendations (high-level follow-up, low-level follow-up, no follow-up). Results: On the basis of reviews during case selection, five nodules seen at follow-up were judged not to have been present at baseline; for 19 of the remaining 95 cases, at least one reader judged the nodule not to have been present at baseline. For the 76 nodules that were unanimously considered to have been present at baseline, 21%-47% (mean 6 standard deviation, 30% +/- 9) were judged to have grown. The kappa values were similar for growth (kappa = 0.55) and a positive screening result (kappa = 0.51) and were lower for a change in margins and attenuation (kappa = 0.27-0.31). The kappa value in the recommendation of high-versus low-level follow-up was high (kappa = 0.66). Conclusion: Reader agreement on nodule growth and screening result was moderate to substantial. Agreement on follow-up recommendations was lower. (C) RSNA, 2011