Feature selection for high-dimensional regression via sparse LSSVR based on Lp-norm

When solving many regression problems, there exist a large number of input features. However, not all features are relevant for current regression, and sometimes, including irrelevant features may deteriorate the learning performance. Therefore, it is essential to select the most relevant features, especially for high-dimensional regression. Feature selection is an effective way to solve this problem. It tries to represent original data by extracting relevant features that contain useful information. In this paper, aiming to effectively select useful features in least squares support vector regression (LSSVR), we propose a novel sparse LSSVR based on L-p-norm (SLSSVR), 0 < p <= 1. Different from the existing L-1-norm LSSVR (L-1-LSSVR) and L-p-norm LSSVR (L-p-LSSVR), SLSSVR uses a smooth approximation of the nonsmooth nonconvex L-p-norm term along with an effective solving algorithm. The proposed algorithm avoids the singularity issue that may encounter in L-p-LSSVR, and its convergency is also guaranteed. Experimental results support the effectiveness of SLSSVR on both feature selection ability and regression performance.

Automated summary

Feature selection is crucial for solving high-dimensional regression problems by extracting relevant features containing useful information to improve learning performance. The sparse LSSVR based on L-p-norm offers an effective method for feature selection, avoiding singularity issues and ensuring convergence. Experimental results demonstrate the effectiveness of SLSSVR in both feature selection ability and regression performance.