An improved multiple fuzzy NNC system based on mutual information and fuzzy integral

Multiple nearest neighbor classifier system (MNNCS) is a popular method to relax the curse of dimensionality. In previous work, most of the MNNCSs are designed by random methods. Random methods may generate unstable component classifiers. In order to relax the randomness, large amount of component classifiers are needed. This paper first extends nearest neighbor classifier into fuzzy nearest neighbor classifier, and proposes a new multiple fuzzy nearest neighbor classifier system based on mutual information and fuzzy integral, called MIFI-MFNNCS. MIFI-MFNNCS adopts target perturbation. Target perturbation decomposes the original classification problem into several sub-problems, where one sub-problem represents one class data. Each sub-problem is described by the relevant data and features. Then it is classified by one component classifier. Therefore, the number of component classifiers can be fixed and reduced. For one component classifier, data may be selected according to its class. And feature is needed to be selected by mutual information. Mutual information can reduce the uncertainty of each component classifier. Feature selection by mutual information in MIFI-MFNNCS may be less affected by the interaction among different classes. The diversity decisions from sub-problem classifiers are combined by fuzzy integral to get the final decision. Here we propose a new method to compute density value according to mutual information, which is a simple method. To demonstrate the performance of the proposed MIFI-MFNNCS, we perform experimental comparisons using five UCI datasets. The results of component classifiers in MIFI-MFNNCS for Ionosphere are shown and analyzed. MIFI-MFNNCS is compared with (1) NNC (2) NNC after feature selection by mutual information (MI-FS-NNC). In multiple fuzzy nearest neighbor classifier system (MFNNCS), mutual information is compared with attribute bagging. And three combination methods are compared, including fuzzy integral, majority voting rule and average. The experimental results show that the accuracy of MIFI-MFNNCS is better than other methods. And mutual information is superior to attribute bagging. Fuzzy integral shows a better performance than majority voting rule and average.