MABUSE: A margin optimization based feature subset selection algorithm using boosting principles

Feature subset selection is one of the most common procedures in machine learning tasks. In a broad sense, feature selection methods can be classified into three major groups, embedded, filter and wrapper methods. Although wrappers might attain superior classification performances, they suffer from scalability issues as they are more computationally expensive than the other methods. Filters are typically faster, and sometimes they are the only applicable methods when the datasets are large. In the field of classification, margin optimization has been proven to be an efficient approach for improving the generalization performance of many classification models. Although margins have been used as criteria for feature selection, in most cases, the most advanced methods are wrappers, which suffer from high computational costs and do not outperform the faster algorithms. In this paper, we propose MABUSE, which is a feature selection method that optimizes margins from a filter perspective. We consider a nearest-neighbor margin definition and, borrowing from the strategy of classifier ensemble construction using boosting, we develop a new method that uses a simple heuristic search. Extensive experimental validation demonstrates that our proposed approach outperforms the state-of-the-art algorithms in both classification and reduction, and has a computational cost that is similar to previous algorithms. (C) 2022 The Author(s). Published by Elsevier B.V.

Automated summary

Feature subset selection is a common procedure in machine learning, with methods classified as embedded, filter, and wrapper. Wrappers achieve better classification performance but suffer from scalability issues. Filters are typically faster and more applicable for large datasets. This paper proposes a new method called MABUSE that optimizes margins from a filter perspective for feature selection. Experimental validation shows that it outperforms other algorithms in classification and reduction tasks, with similar computational cost.