A New Accuracy Assessment Method for One-Class Remote Sensing Classification

In one-class remote sensing classification, users are only interested in classifying one specific land type (positive class), without considering other classes (negative class). Previous researchers have proposed different one-class classification methods without requiring negative data. An appropriate accuracy measure is usually needed to tune free parameters/threshold and to evaluate the classification result. However, traditional accuracy measures, such as the kappa coefficient and F-measure (F), require both positive and negative data, and hence, they are not applicable for positive-only data. In this paper, we investigate a new accuracy assessment method that does not require negative data. Two new statistics F-pb (proxy of F-measure based on positive-background data) and F-cpb (prevalence-calibrated proxy of F-measure based on positive-background data) can be calculated from a modified confusion matrix, where the observed negative data are replaced by background data. To investigate the effectiveness of the new method, we produced different one-class classification results using two scenes of aerial photograph, and the accuracy values were evaluated by F-pb, F-cpb, kappa coefficient, and F. The effectiveness of F-pb in model and threshold selection was investigated as well. Experimental results show that the behaviors of F-pb, F-cpb, F, and kappa coefficient are similar, and they all rank the models by accuracy similarly. In model and threshold selection, the classification accuracy values produced by maximizing F-pb and F are similar, and they are higher than those produced by setting an arbitrary rejection fraction. Therefore, we conclude that the new method is effective in model selection, threshold selection, and accuracy assessment, and it will have important applications in one-class remote sensing classification since negative data are not needed.