期刊
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
卷 30, 期 11, 页码 3338-3346出版社
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TNNLS.2019.2891088
关键词
Convergence; Acceleration; Machine learning algorithms; Machine learning; Optimization; Neural networks; Learning systems; Limited memory version of Broyden-Fletcher-Goldfarb-Shanno (L-BFGS); machine learning; Nesterov's acceleration; optimization method; variance reduction
类别
资金
- National Natural Science Foundation of China [61673179]
- Fundamental Research Funds for the Central Universities
The limited memory version of the Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) algorithm is the most popular quasi-Newton algorithm in machine learning and optimization. Recently, it was shown that the stochastic L-BFGS (sL-BFGS) algorithm with the variance-reduced stochastic gradient converges linearly. In this paper, we propose a new sL-BFGS algorithm by importing a proper momentum. We prove an accelerated linear convergence rate under mild conditions. The experimental results on different data sets also verify this acceleration advantage.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据