Adaptiveness and consistency of a class of online ensemble learning algorithms

Expert based ensemble learning algorithms often serve as online learning algorithms for an unknown, possibly time-varying, probability distribution. Their simplicity allows flexibility in design choices, leading to variations that balance adaptiveness and consistency. This article provides an analytical framework to quantify the adaptiveness and consistency of expert based ensemble learning algorithms. With properly selected states, the algorithms are modeled as a Markov chains. Then quantitative metrics of adaptiveness and consistency can be calculated through mathematical formulas, other than relying on numerical simulations. Results are derived for several popular ensemble learning algorithms. Success of the method has also been demonstrated in both simulation and experimental results.

Automated summary

This article introduces an analytical framework to quantify the adaptiveness and consistency of expert based ensemble learning algorithms. By modeling the algorithms as Markov chains with properly selected states, quantitative metrics can be calculated through mathematical formulas, instead of relying on numerical simulations. Results for popular ensemble learning algorithms have been derived and the success of the method has been demonstrated in simulation and experimental results.