A stochastic contraction mapping theorem

In this paper we define contractive and nonexpansive properties for adapted stochastic processes which can be used to deduce limiting properties. In general, nonexpansive processes possess finite limits while contractive processes converge to zero a.e. A general method for obtaining convergence rates is proposed. These techniques are applied to a number of important processes, including stochastic approximation, least-squares estimation of controlled linear models and Q-learning. (c) 2023 Elsevier B.V. All rights reserved.

Automated summary

In this paper, contractive and nonexpansive properties for adapted stochastic processes are defined, which can be used to analyze limiting properties. Nonexpansive processes generally have finite limits, while contractive processes converge to zero almost everywhere. A general method for obtaining convergence rates is proposed. These techniques are applied to various important processes, including stochastic approximation, least-squares estimation of controlled linear models, and Q-learning.