A hybrid regularization approach for random vector functional-link networks

Neural networks have been widely applied to expert and intelligent systems in the fields of business, fault diagnosis, and forecasting. Especially, random vector functional-link networks (RVFL), an important structure, have also been considerably studied and used in recent years. This paper addresses in the investigation of regularization model for RVFL, and proposes a hybrid regularization approach for RVFL by adding the l(2) and l(1) norm penalty terms simultaneously. A novel iterative learning algorithm is developed by using a fixed point contractive map. Further, some theoretical properties, including the convergence, sparsity, and stability of the proposed algorithm, are discussed and analyzed concretely under reasonable assumptions. The proposed method greatly improves the learner's sparsity and stability, guaranteeing the feasibility and effectiveness of network training. Experimental results on some benchmarks as well as face recognition database collected from the expert and intelligent systems, particularly the use of the statistical analysis strategy, verify the effectiveness and superiority of the proposed method, i.e. this new algorithm systematically outperforms the original RVFL and its variants in terms of the accuracy, sparsity, and stability of the solution. (C) 2019 Elsevier Ltd. All rights reserved.