Regression and classification using extreme learning machine based on L1-norm and L2-norm

Extreme learning machine (ELM) is a very simple machine learning algorithm and it can achieve a good generalization performance with extremely fast speed. Therefore it has practical significance for data analysis in real-world applications. However, it is implemented normally under the empirical risk minimization scheme and it may tend to generate a large-scale and over-fitting model. In this paper, an ELM model based on L-1-norm and L-2-norm regularizations is proposed to handle regression and multiple-class classification problems in a unified framework. The proposed model called L-1-L-2-ELM combines the grouping effect benefits of L-2 penalty and the tendency towards sparse solution of L-1 penalty, thus it can control the complexity of the network and prevent over-fitting. To solve the mixed penalty problem, the separate elastic net algorithm and Bayesian information criterion (BIC) are adopted to find the optimal model for each response variable. We test the L-1-L-2-ELM algorithm on one artificial case and nine benchmark data sets to evaluate its performance. Simulation results have shown that the proposed algorithms outperform the original ELM as well as other advanced ELM algorithms in terms of prediction accuracy, and it is more robust in both regression and classification applications. (C) 2015 Elsevier B.V. All rights reserved.