ON THE OPTIMAL TRANSITION MATRIX FOR MARKOV CHAIN MONTE CARLO SAMPLING

Let chi be a finite space and let pi be an underlying probability on chi. For any real-valued function f defined on chi, we are interested in calculating the expectation of f under pi. Let X-0, X-1,..., X-n,... be a Markov chain generated by some transition matrix P with invariant distribution pi. The time average, 1/n Sigma(n-1)(k=0) f(X-k), is a reasonable approximation to the expectation, E-pi[f(X)]. Which matrix P minimizes the asymptotic variance of 1/n Sigma(n-1)(k=0) f(X-k)? The answer depends on f. Rather than a worst-case analysis, we will identify the set of P's that minimize the average asymptotic variance, averaged with respect to a uniform distribution on f.