UNBIASED MLMC-BASED VARIATIONAL BAYES FOR LIKELIHOOD-FREE INFERENCE

Variational Bayes (VB) is a popular tool for Bayesian inference in statistical modeling. Recently, some VB algorithms were proposed to handle intractable likelihoods with applications such as approximate Bayesian computation. In this paper, we propose several unbiased estimators based on multilevel Monte Carlo (MLMC) for the gradient of Kullback-Leibler divergence between the posterior distribution and the variational distribution when the likelihood is intractable, but can be estimated unbiasedly. The new VB algorithm differs from the VB algorithms in the literature which usually render biased gradient estimators. Moreover, we incorporate randomized quasi-Monte Carlo (RQMC) sampling within the MLMC-based gradient estimators, which was known to provide a favorable rate of convergence in numerical integration. Theoretical guarantees for RQMC are provided in this new setting. Numerical experiments show that using RQMC in MLMC greatly speeds up the VB algorithm, and finds a better parameter value than some existing competitors do.

Automated summary

This paper proposes a new variational Bayes algorithm based on multilevel Monte Carlo and randomized quasi-Monte Carlo sampling, for handling inference problems with intractable likelihood functions. Compared to existing algorithms, this algorithm has better performance and convergence rate.