The Papanicolaou Society of Cytopathology guidelines for respiratory cytology: Reproducibility of categories among observers

Introduction: The Papanicolaou Society of Cytopathology (PSC) has developed a set of guidelines for reporting respiratory cytology. While the malignancy risk for each category is known, the interobserver reproducibility of these diagnostic categories has not been well described. Methods: Fifty-five cytologic specimens obtained by fine needle aspiration from the pulmonary nodules were independently reviewed by four board-certified cytopathologists and assigned to the diagnostic categories described by the PSC guidelines for respiratory specimens. Statistical analysis for diagnostic accuracy was performed for absolute agreement and chance-corrected agreement (kappa). Differences in frequency of distribution of diagnoses between raters were assessed using the Kruskal-Wallis test. Results: No significant differences in distribution of scores by raters were observed. On average, the absolute agreement was 49.5% and the chance-corrected agreement (kappa) was 20%. 34.5% of interrater comparisons were in full agreement, and total lack of agreement between the four categories was found in 3% of cases. Combining the suspicious for malignancy category with the malignant category did not significantly alter interrater agreement statistics. Conclusions: The PSC categories showed only fair reproducibility among four cytopathologists. Agreement between raters was at best fair and did not improve significantly when the categories suspicious for malignancy and malignant were combined. The most common source of disagreement appeared to be between the categories suspicious and malignant.