期刊
COMPUTATIONAL OPTIMIZATION AND APPLICATIONS
卷 76, 期 2, 页码 347-380出版社
SPRINGER
DOI: 10.1007/s10589-020-00183-1
关键词
Incremental aggregated gradient method; Curvature information; Nesterov accelerated gradient method
This paper studies an acceleration technique for incremental aggregated gradient (IAG) method through the use of curvature information for solving strongly convex finite sum optimization problems. These optimization problems of interest arise in large-scale learning applications. Our technique utilizes a curvature-aided gradient tracking step to produce accurate gradient estimates incrementally using Hessian information. We propose and analyze two methods utilizing the new technique, the curvature-aided IAG (CIAG) method and the accelerated CIAG (A-CIAG) method, which are analogous to gradient method and Nesterov's accelerated gradient method, respectively. Setting kappa to be the condition number of the objective function, we prove the R linear convergence rates of 1-4c0 kappa(kappa+1)2for the CIAG method, and 1-c12 kappa for the A-CIAG method, where c0,c1 <= 1 are constants inversely proportional to the distance between the initial point and the optimal solution. When the initial iterate is close to the optimal solution, the R linear convergence rates match with the gradient and accelerated gradient method, albeit CIAG and A-CIAG operate in an incremental setting with strictly lower computation complexity. Numerical experiments confirm our findings. The source codes used for this paper can be found on .
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据