An investigation into the effects of label noise on Dynamic Selection algorithms

In the literature on classification problems, it is widely discussed how the presence of label noise can bring about severe degradation in performance. Several works have applied Prototype Selection techniques, Ensemble Methods, or both, in an attempt to alleviate this issue. Nevertheless, these methods are not always able to sufficiently counteract the effects of noise. In this work, we investigate the effects of noise on a particular class of Ensemble Methods, that of Dynamic Selection algorithms, and we are especially interested in the behavior of the Fire-DES++ algorithm, a state of the art algorithm which applies the Edited Nearest Neighbors (ENN) algorithm to deal with the effects of noise and imbalance. We propose a method which employs multiple Dynamic Selection sets, based on the Bagging-IH algorithm, which we dub Multiple-Set Dynamic Selection (MSDS), in an attempt to supplant the ENN algorithm on the filtering step. We find that almost all methods based on Dynamic Selection are severely affected by the presence of label noise, with the exception of the K-Nearest Oracles-Union algorithm. We also find that our proposed method can alleviate the issues caused by noise in some scenarios. We have made the code for our method available at https://github.com/fnw/baggingds.

Automated summary

In the literature on classification problems, the impacts of label noise on performance is widely discussed, however current methods are not always effective in combating noise. This study investigates the effects of noise on dynamic selection algorithms, proposing the use of Multiple-Set Dynamic Selection method to supplant the ENN algorithm, and finds that the K-Nearest Oracles-Union algorithm is the only method unaffected by noise.