Performance of global-local hybrid ensemble versus boosting and bagging ensembles

This study compares the classification performance of a hybrid ensemble, which is called the global- local hybrid ensemble that employs both local and global learners against data manipulation ensembles including bagging and boosting variants. A comprehensive simulation study is performed on 46 UCI machine learning repository data sets using prediction accuracy and SAR performance metrics and along with rigorous statistical significance tests. Simulation results for comparison of classification performances indicate that global-local hybrid ensemble outperforms or ties with bagging and boosting ensemble variants in all cases. This suggests that the global-local ensemble has a more robust performance profile since its performance is less sensitive to variation with respect to the problem domain, or equivalently the data sets. This performance robustness is realized at the expense of increased complexity of the global-local ensemble since at least two types of learners, e.g. one global and another one local, must be trained. A complementary diversity analysis of global-local hybrid ensemble and base learners used for bagging and boosting ensembles on select data sets in the classifier projection space provides both an explanation and support for the performance related findings of this study.