Comments on Anderson Acceleration, Mixing and Extrapolation

The Extrapolation Algorithm is a technique devised in 1962 for accelerating the rate of convergence of slowly converging Picard iterations for fixed point problems. Versions to this technique are now called Anderson Acceleration in the applied mathematics community and Anderson Mixing in the physics and chemistry communities, and these are related to several other methods extant in the literature. We seek here to broaden and deepen the conceptual foundations for these methods, and to clarify their relationship to certain iterative methods for root-finding problems. For this purpose, the Extrapolation Algorithm will be reviewed in some detail, and selected papers from the existing literature will be discussed, both from conceptual and implementation perspectives.