The added value of an artificial intelligence system in assisting radiologists on indeterminate BI-RADS 0 mammograms

Objectives To investigate the value of an artificial intelligence (AI) system in assisting radiologists to improve the assessment accuracy of BI-RADS 0 cases in mammograms. Methods We included 34,654 consecutive digital mammography studies, collected between January 2011 and January 2019, among which, 1088 cases from 1010 unique patients with initial BI-RADS 0 assessment who were recalled during 2 years of follow-up were used in this study. Two mid-level radiologists retrospectively re-assessed these BI-RADS 0 cases with the assistance of an AI system developed by us previously. In addition, four entry-level radiologists were split into two groups to cross-read 80 cases with and without the AI. Diagnostic performance was evaluated using the follow-up diagnosis or biopsy results as the reference standard. Results Of the 1088 cases, 626 were actually normal (BI-RADS 1 and no recall required). Assisted by the AI system, 351 (56%) and 362 (58%) normal cases were correctly identified by the two mid-level radiologists hence can be avoided for unnecessary follow-ups. However, they would have missed 12 (10 invasive cancers and 2 ductal carcinoma in situ cancers) and 6 (invasive cancers) malignant lesions respectively as a result. These missed lesions were not highly malignant tumors. The inter-rater reliability of entry-level radiologists increased from 0.20 to 0.30 (p < 0.005) by introducing the AI. Conclusion The AI system can effectively assist mid-level radiologists in reducing unnecessary follow-ups of mammographically indeterminate breast lesions and reducing the benign biopsy rate without missing highly malignant tumors.

Automated summary

The study investigated the value of an artificial intelligence system in assisting radiologists to improve the assessment accuracy of BI-RADS 0 cases in mammograms. The AI system was found to effectively reduce unnecessary follow-ups, decrease benign biopsy rates, and not miss highly malignant tumors.