Using spectral entropy and bernoulli map to handle concept drift

Data stream mining is a relevant task to extract information from large amounts of data that continuously evolve over time. In this context, learning algorithms may combine a classifier and a drift detector to identify changes in the distribution of the predictions error in order to rapidly adapt or replace the predictive model. Several proposals have been presented in the literature for the detection of concept changes based on the error rate of the predictive models. In general, the error rate distribution grounds most of the approaches based on sequential analysis and statistical process control, or by monitoring distributions using sliding windows, which assume the prediction errors are generated independently. However, empirical studies have shown that the error rate can be influenced by temporal dependence. In addition, new approaches considering dynamical system tools have been proposed for concept drift detection in unsupervised scenarios containing temporal dependencies. Motivated by these approaches, this article proposes the Spectral Entropy Drift Detector (SEDD), which is based on Spectral Entropy, Bernoulli Map and on the surrogate stability concept. Experimental results using abrupt and gradual concept drift versions of different dataset generators as well as real-world data streams, run in the Massive Online Analysis (MOA) framework, suggest that SEDD was competitive with the state-of-the-art methods, especially when considering accuracy and false alarms.

Automated summary

Data stream mining involves extracting information from continuously evolving large amounts of data, where learning algorithms can combine a classifier and drift detector to identify changes in prediction error distribution. While many methods are based on error rate distribution, empirical studies have shown that error rate can be influenced by temporal dependence. New approaches, including using dynamical system tools, have been proposed for concept drift detection in unsupervised scenarios with temporal dependencies.