ρ-regularization subproblems: strong duality and an eigensolver-based algorithm

Trust-region (TR) type method, based on a quadratic model such as the trust-region subproblem (TRS) and p-regularization subproblem (pRS), is arguably one of the most successful methods for unconstrained minimization. In this paper, we study a general regularized subproblem (named rho RS), which covers TRS and pRS as special cases. We derive a strong duality theorem for rho RS, and also its necessary and sufficient optimality condition under general assumptions on the regularization term. We then define the Rendl-Wolkowicz (RW) dual problem of rho RS, which is a maximization problem whose objective function is concave, and differentiable except possibly at two points. It is worth pointing out that our definition is based on an alternative derivation of the RW-dual problem for TRS. Then we propose an eigensolver-based algorithm for solving the RW-dual problem of rho RS. The algorithm is carried out by finding the smallest eigenvalue and its unit eigenvector of a certain matrix in each iteration. Finally, we present numerical results on randomly generated rho RS's, and on a new class of regularized problem that combines TRS and pRS, to illustrate our algorithm.

Automated summary

This paper studies TR type methods and proposes a general regularized subproblem. It derives a strong duality theorem and necessary and sufficient optimality condition for the subproblem. An eigensolver-based algorithm is also proposed to solve the derived dual problem. Numerical results are presented to validate the effectiveness of the algorithm.