Computer-aided diagnosis to distinguish benign from malignant solitary pulmonary nodules on radiographs: ROC analysis of radiologists'

PURPOSE: To evaluate radiologists' performance for determining a distinction between benign and malignant pulmonary nodules on chest radiographs without and with use of a computer-aided diagnosis scheme. MATERIALS AND METHODS: Fifty-three chest radiographs that depicted 31 primary lung cancers and 22 benign nodules were used. The likelihood measure of malignancy for each nodule was determined by using an automated computerized scheme. Sixteen radiologists (nine attending radiologists and seven radiology residents) participated in an observer study in which cases were interpreted first without. and then with use of the scheme. The radiologists' performance was evaluated with receiver operating characteristic analysis. RESULTS: The mean area under the best-fit binormal receiver operating characteristic curve plotted in the unit square (A(z)) values of radiologists who interpreted images without and with the scheme were 0.743 and 0.817, respectively. The performance of radiologists was improved significantly when the scheme was used (p =.002). However, the performance (A(z) = 0.889) of the computer alone exceeded these results by a substantial margin. The average change in radiologists' confidence level for interpretation without and with the scheme was highly correlated (r = 0.845) with the likelihood measure of malignancy, which was presented as computer output. CONCLUSION: This scheme for computer-aided diagnosis has the potential to improve the accuracy of radiologists' performance in the classification of benign and malignant solitary pulmonary nodules.