Interreader Variability of Prostate Imaging Reporting and Data System Version 2 in Detecting and Assessing Prostate Cancer Lesions at Prostate MRI

OBJECTIVE. The purpose of this study was to evaluate agreement among radiologists in detecting and assessing prostate cancer at multiparametric MRI using Prostate Imaging Reporting and Data System version 2 (PI-RADSv2). MATERIALS AND METHODS. Treatment-naive patients underwent 3-T multiparametric MRI between April 2012 and June 2015. Among the 163 patients evaluated, 110 underwent prostatectomy after MRI and 53 had normal MRI findings and transrectal ultrasound-guided biopsy results. Nine radiologists participated (three each with high, intermediate, and low levels of experience). Readers interpreted images of 58 patients on average (range, 56-60) using PI-RADSv2. Prostatectomy specimens registered to MRI were ground truth. Interobserver agreement was evaluated with the index of specific agreement for lesion detection and kappa and proportion of agreement for PI-RADS category assignment. RESULTS. The radiologists detected 336 lesions. Sensitivity for index lesions was 80.9% (95% CI, 75.1-85.9%), comparable across reader experience (p = 0.392). Patient-level specificity was experience dependent; highly experienced readers had 840% specificity versus 55.2% for all others (p < 0.001). Interobserver agreement was excellent for detecting index lesions (index of specific agreement, 0.871; 95% CI, 0.798-0.923). Agreement on PI-RADSv2 category assignment of index lesions was moderate (kappa = 0.419; 95% CI, 0.238-0.595). For individual category assignments, proportion of agreement was slight for PI-RADS category 3 (0.208; 95% CI, 0.086-0.284) but substantial for PI-RADS category 4 (0.674; 95% CI, 0.540-0.776). However, proportion of agreement for T2-weighted PI-RADS 4 in the transition zone was 0.250 (95% CI, 0.108-0.372). Proportion of agreement for category assignment of index lesions on dynamic contrast-enhanced MR images was 0.822 (95% CI, 0.728-0.903), on T2-weighted MR images was 0.515 (95% CI, 0.430-0623), and on DW images was 0.586 (95% CI, 0.495-0.682). Proportion of agreement for dominant lesion was excellent (0.828; 95% CI, 0.742-0.913). CONCLUSION. Radiologists across experience levels had excellent agreement for detecting index lesions and moderate agreement for category assignment of lesions using PI-RADS. Future iterations of PI-RADS should clarify PI-RADS 3 and PI-RADS 4 in the transition zone.