To reject or not to reject: That is the question - An answer in case of neural classifiers

In this paper a method defining a reject option applicable to a given ii-reject classifier is proposed. The reject option is based on an estimate of the classification reliability; measured by a reliability evaluator Psi. Trivially once a reject threshold sigma has been fixed. a sample is rejected if the corresponding value of Psi is below sigma. Obviously, as sigma represents the least tolerable classification reliability level, when its value varies the reject option becomes more or less severe. In order to adapt the behavior of the reject option to the requirements of the considered application domain, a function P characterizing the reject option's adequacy to the domain has been introduced. It is shown that P can be expressed as a function of sigma and, consequently, the optimal value for sigma is defined as the one which maximizes the function P. The method for determining the optimal threshold value is independent of the specific 0-reject classifier, while the definition of the reliability evaluators is related to the classifier's architecture, General criteria for defining appropriate reliability evaluators within a classification paradigm are illustrated in the paper and are based on the localization, in the feature space, of the samples that could be classified with a low reliability. The definition of the reliability evaluators for three popular architectures of neural networks. back-propagation, learning vector quantization and probabilistic neural network, is presented. Finally, the method has been tested with reference to a complex classification problem with data generated according to a distribution of distribution model.