Coupling-based convergence assessment of some Gibbs samplers for high-dimensional Bayesian regression with shrinkage priors

We consider Markov chain Monte Carlo (MCMC) algorithms for Bayesian high-dimensional regression with continuous shrinkage priors. A common challenge with these algorithms is the choice of the number of iterations to perform. This is critical when each iteration is expensive, as is the case when dealing with modern data sets, such as genome-wide association studies with thousands of rows and up to hundreds of thousands of columns. We develop coupling techniques tailored to the setting of high-dimensional regression with shrinkage priors, which enable practical, non-asymptotic diagnostics of convergence without relying on traceplots or long-run asymptotics. By establishing geometric drift and minorization conditions for the algorithm under consideration, we prove that the proposed couplings have finite expected meeting time. Focusing on a class of shrinkage priors which includes the 'Horseshoe', we empirically demonstrate the scalability of the proposed couplings. A highlight of our findings is that less than 1000 iterations can be enough for a Gibbs sampler to reach stationarity in a regression on 100,000 covariates. The numerical results also illustrate the impact of the prior on the computational efficiency of the coupling, and suggest the use of priors where the local precisions are Half-t distributed with degree of freedom larger than one.

Automated summary

This paper considers the use of Markov chain Monte Carlo (MCMC) algorithms with continuous shrinkage priors in Bayesian high-dimensional regression. The authors address the challenge of determining the number of iterations to perform, particularly in the context of modern data sets with large numbers of covariates. They propose coupling techniques tailored to this setting, allowing for practical, non-asymptotic diagnostics of convergence. By establishing conditions for drift and minorization, they prove that the proposed couplings have finite expected meeting time. Empirical results demonstrate the scalability of the proposed couplings, with less than 1000 iterations being sufficient for a Gibbs sampler to reach stationarity in a regression with 100,000 covariates.