Reliability of the BI-RADS Final Assessment Categories and Management Recommendations in a Telemammography Context

Purpose: The aim of this study was to evaluate the intradevice and interdevice reliability of four alternatives for telemammography-computed radiography, printed film, a film digitizer, and a digital camera-in terms of interpretation agreement when using the BI-RADS (R) lexicon. Methods: The ethics committee of the authors' institution approved this retrospective study. A factorial design with repeated measures with 1,960 interpretations was used (70 patients, seven radiologists, and four devices). Reliability was evaluated using the kappa coefficient for intradevice and interdevice agreement on malignancy classification and on BI-RADS final assessment category. Results: Agreement on malignancy classification was higher than agreement for BI-RADS final assessment category. Interdevice agreement on malignancy classification between the film digitizer and computed radiography was ranked as almost perfect (P < .001), whereas interdevice agreement for the other alternatives was ranked as substantial (P < .001), with observed agreement ranging from 85% to 91% and kappa values ranging from 0.70 to 0.81. Interdevice agreement on BI-RADS final assessment category was ranked as substantial or moderate (P < .001), with observed agreement ranging from 64% to 77% and kappa values ranging from 0.52 to 0.69. Interdevice agreement was higher than intradevice agreement. Conclusions: The results of this study show very high interdevice agreement, especially for management recommendations derived from malignancy classification, which is one of the most important outcomes in screening programs. This study provides evidence to suggest the interchangeability of the devices evaluated, thereby enabling the provision of low-cost medical imaging services to underserved populations.