Over-relaxation of the fast iterative shrinkage-thresholding algorithm with variable stepsize

In this paper, we present an over-relaxed variant of the fast iterative shrinkage-thresholding algorithm (FISTA)/the monotone FISTA (MFISTA). FISTA and MFISTA are iterative first-order algorithms, whose convergence rates of the objective function are O(1/k(2)) for an iteration counter k, for the minimization of the sum of a smooth and a nonsmooth convex function. FISTA and MFISTA are composed of the forward-backward splitting step together with a certain computationally efficient shifting step. The stepsize available in the forward-backward splitting step in these algorithms has been limited to a fixed value determined by the Lipschitz constant of the gradient of the smooth function. Examples of the proposed scheme admit variable stepsizes in broader ranges than FISTA/MFISTA, while keeping the same convergence rate O(1/k(2)). A numerical example in a well-conditioned case demonstrates the effect of the proposed relaxations by showing that the proposed scheme outperforms, in the speed of convergence, the original FISTA and MFISTA.