Regression random machines: An ensemble support vector regression modelwith free kernel choice

Machine learning techniques have one of their main objectives to reduce the generalized prediction error.Support vector models have as a main challenge the choice of an appropriate kernel function, as well asthe estimation of its hyperparameters. Such procedures are usually performed through some tests and tuningprocesses which require a high computational performance. In contrast, ensemble methods present a goodapproach to combine several models which result in a greater predictive capacity. In this paper, we propose anew ensemble method to support vector regression, namely regression random machines. The proposed methodeliminates the need to choose the best kernel function during the tuning process using a random mixture ofkernel functions combined with a properly bagging ensemble which considers the strength and agreement ofthe single models. The results demonstrated a good predictive performance through lower generalization errorwhich overlaps the single and bagged versions of support vector models with different kernels. The usefulnessof the proposed method is illustrated by simulation studies that were realized over eight artificial scenariosand twenty-seven real-world applications

Automated summary

This paper proposes a new ensemble method called regression random machines for support vector regression. By using a random mixture of kernel functions and a properly bagging ensemble, this method eliminates the need to choose the best kernel function during the tuning process and achieves good predictive performance.