Assessing a single classification accuracy measure to deal with the imprecision of location and class: Fuzzy weighted Kappa versus Kappa

Standard accuracy assessment in raster modelling assumes that desirable classes are exhaustively exclusive, location error either does not exist or does not affect classification, and classification results are crisp, not fuzzy. Failing to take these into account leads to propagation of the error. This paper introduces a fuzzy weighted Kappa measure as I global statistic which takes both of these Sources of uncertainty into account in classification accuracy assessment. Calculation of the fuzzy weighted Kappa measure involves three processes. First, a Virtual confusion matrix is constructed to incorporate the fuzziness of location. Then a similarity weight matrix is derived to represent the fuzziness of the category. Combining the virtual confusion matrix and the similarity weight matrix, the fuzzy weighted Kappa is calculated using the weighted Kappa algorithms. The fuzzy weighted Kappa is shown to be more flexible than standard methods and gives results closer to an intuitive assessment of accuracy. Although calculating the fuzzy weighted Kappa involves a degree of subjectivity, this poses no major limitations on the usefulness of the measure. It is therefore proposed as an alternative classification accuracy measure for the remote sensing and GISc community.