A proportional-integral-derivative-incorporated stochastic gradient descent-based latent factor analysis model

Large-scale relationships like user-item preferences in a recommender system are mostly described by a high-dimensional and sparse (HiDS) matrix. A latent factor analysis (LFA) model extracts useful knowledge from an HiDS matrix efficiently, where stochastic gradient descent (SGD) is frequently adopted as the learning algorithm. However, a standard SGD algorithm updates a decision parameter with the stochastic gradient on the instant loss only, without considering information described by prior updates. Hence, an SGD-based LFA model commonly consumes many iterations to converge, which greatly affects its practicability. On the other hand, a proportional-integral-derivative (PID) controller makes a learning model converge fast with the consideration of its historical errors from the initial state till the current moment. Motivated by this discovery, this paper proposes a PID-incorporated SGD-based LFA (PSL) model. Its main idea is to rebuild the instant error on a single instance following the principle of PID, and then substitute this rebuilt error into an SGD algorithm for accelerating model convergence. Empirical studies on six widely-accepted HiDS matrices indicate that compared with state-of-the-art LFA models, a PSL model achieves significantly higher computational efficiency as well as highly competitive prediction accuracy for missing data of an HiDS matrix. (c) 2020 Elsevier B.V. All rights reserved.

Automated summary

The proposed PID-incorporated SGD-based LFA (PSL) model accelerates model convergence by rebuilding instant errors based on the PID principle. Empirical studies show that the PSL model achieves significantly higher computational efficiency and competitive prediction accuracy for missing data in high-dimensional and sparse matrices compared to state-of-the-art LFA models.