A Survey of Ensemble Learning: Concepts, Algorithms, Applications, and Prospects

Ensemble learning techniques have achieved state-of-the-art performance in diverse machine learning applications by combining the predictions from two or more base models. This paper presents a concise overview of ensemble learning, covering the three main ensemble methods: bagging, boosting, and stacking, their early development to the recent state-of-the-art algorithms. The study focuses on the widely used ensemble algorithms, including random forest, adaptive boosting (AdaBoost), gradient boosting, extreme gradient boosting (XGBoost), light gradient boosting machine (LightGBM), and categorical boosting (CatBoost). An attempt is made to concisely cover their mathematical and algorithmic representations, which is lacking in the existing literature and would be beneficial to machine learning researchers and practitioners.

Automated summary

Ensemble learning techniques have achieved state-of-the-art performance by combining predictions from multiple base models, with a focus on widely used algorithms such as random forest, AdaBoost, gradient boosting, XGBoost, LightGBM, and CatBoost. This overview aims to provide concise coverage of their mathematical and algorithmic representations, lacking in existing literature, for the benefit of machine learning researchers and practitioners.