Contraction rates for sparse variational approximations in Gaussian process regression

We study the theoretical properties of a variational Bayes method in the Gaussian Process regression model. We consider the inducing variables method introduced by Titsias (2009b) and derive sufficient conditions for obtaining contraction rates for the corresponding vari-ational Bayes (VB) posterior. As examples we show that for three particular covariance kernels (Matern, squared exponential, random series prior) the VB approach can achieve optimal, minimax contraction rates for a sufficiently large number of appropriately chosen inducing variables. The theoretical findings are demonstrated by numerical experiments.

Automated summary

This study investigates the theoretical properties of the variational Bayes method in the Gaussian Process regression model. By considering the inducing variables method introduced by Titsias (2009b), we derive sufficient conditions for obtaining contraction rates for the corresponding variational Bayes posterior. Numerical experiments demonstrate the validity of the theoretical findings, showing that the VB approach can achieve optimal contraction rates for certain covariance kernels.