Sparse principal component regression via singular value decomposition approach

Principal component regression (PCR) is a two-stage procedure: the first stage performs principal component analysis (PCA) and the second stage builds a regression model whose explanatory variables are the principal components obtained in the first stage. Since PCA is performed using only explanatory variables, the principal components have no information about the response variable. To address this problem, we present a one-stage procedure for PCR based on a singular value decomposition approach. Our approach is based upon two loss functions, which are a regression loss and a PCA loss from the singular value decomposition, with sparse regularization. The proposed method enables us to obtain principal component loadings that include information about both explanatory variables and a response variable. An estimation algorithm is developed by using the alternating direction method of multipliers. We conduct numerical studies to show the effectiveness of the proposed method.

Automated summary

Principal component regression (PCR) is a two-stage procedure involving principal component analysis (PCA) and regression modeling. To address the issue of principal components lacking information about the response variable, a one-stage PCR procedure based on singular value decomposition with sparse regularization is proposed. The method allows for obtaining principal component loadings that include information from both explanatory variables and the response variable, and an estimation algorithm using the alternating direction method of multipliers is developed to demonstrate its effectiveness through numerical studies.