On adaptive Markov chain Monte Carlo algorithms

We look at adaptive Markov chain Monte Carlo algorithms that generate stochastic processes based on sequences of transition kernels, where each transition kernel is allowed to depend on the history of the process. We show under certain conditions that the stochastic process generated is ergodic, with appropriate stationary distribution. We use this result to analyse an adaptive version of the random walk Metropolis algorithm where the scale parameter sigma is sequentially adapted using a Robbins-Monro type algorithm in order to find the optimal scale parameter sigma(opt). We close with a simulation example.