On the convergence of the decomposition method for support vector machines

The decomposition method is currently one of the major methods for solving support vector machines (SVMs). Its convergence properties have not been fully understood. The general asymptotic convergence was first proposed by Chang et al. However, their working set selection does not coincide with existing implementation. A later breakthrough by Keerthi and Gilbert proved the convergence finite termination for practical cases while the size of the working set is restricted to two. In this paper, we prove the asymptotic convergence of the algorithm used by the software SVMlight and other later implementation. The size of the working set can be any even number. Extensions to other SVM formulations are also discussed.