Optimizing weighted lazy learning and Naive Bayes classification using differential evolution algorithm

Naive Bayes is a well-known classification algorithm in machine learning area. Since it is based on combinations of efficiently computable probability estimates, NB is often a good choice for high-dimensional problems such as text classification. However, NB is based on a conditional independence assumption between attributes which is often violated in real-world applications. Accordingly, lots of work has been done to improve the performance of NB, such as structure extension, attribute selection, attribute weighting, instance weighting and instance selection. An alternative strategy to address the limitations of NB is to apply NB only over the neighbors of the instance that needs to be classified, where the independence assumption may be more justified. However, this introduces another practical problem - the high variance resulting from insufficient training data when building the NB model only over the neighbors rather than the original training dataset. In this paper, a new learning algorithm named Weighted Lazy Naive Bayes (WLNB) is presented. WLNB is designed to address the variance issue by augmenting the nearest neighbors of a test instance. Then, a self-adaptive evolutionary process is applied to automatically learn two key parameters in WLNB. In this way, a method named Evolutionary Weighted Lazy Naive Bayes (EWLNB) is formed. EWLNB uses Differential Evolution to search optimal values for parameters, which makes the method quite effective. Experimental evaluations on 56 UCI machine learning benchmark datasets demonstrate that EWLNB significantly outperforms NB and several other improved NB algorithms in terms of classification accuracy and class probability estimation.

Automated summary

Naive Bayes is a well-known classification algorithm that performs well in high-dimensional problems such as text classification, but its assumption of attribute independence is often violated in real-world applications. Various techniques have been developed to improve the performance of Naive Bayes, with one alternative strategy being to apply Naive Bayes only over neighboring instances to reduce variance.