A modified Armijo rule for the online selection of learning rate of the LMS algorithm

The use of the Armijo rule for the automatic selection of the step size within the class of stochastic gradient descent algorithms is investigated, and the Armijo rule learning rate least mean square (ALR-LMS) algorithm is introduced. This algorithm is derived by integrating an appropriately modified version of the Armijo line search to the least mean square filter update. The analysis of the stability, robustness and the bounds on the parameters which guarantee convergence is conducted, and sonic practical issues relating the choice of parameters of the ALR-LMS and computational complexity are addressed. Comprehensive simulation results in the system identification and prediction setting support the analysis. (C) 2009 Elsevier Inc. All rights reserved.