Pseudo-marginal Bayesian inference for Gaussian process latent variable models

A Bayesian inference framework for supervised Gaussian process latent variable models is introduced. The framework overcomes the high correlations between latent variables and hyperparameters by collapsing the statistical model through approximate integration of the latent variables. Using an unbiased pseudo estimate for the marginal likelihood, the exact hyperparameter posterior can then be explored using collapsed Gibbs sampling and, conditional on these samples, the exact latent posterior can be explored through elliptical slice sampling. The framework is tested on both simulated and real examples. When compared with the standard approach based on variational inference, this approach leads to significant improvements in the predictive accuracy and quantification of uncertainty, as well as a deeper insight into the challenges of performing inference in this class of models.

Automated summary

This Bayesian inference framework for supervised Gaussian process latent variable models introduces a method to collapse the statistical model to overcome high correlations between latent variables and hyperparameters. By using collapsed Gibbs sampling and elliptical slice sampling, the exact hyperparameter posterior and latent posterior can be explored. Compared to variational inference, this approach leads to significant improvements in predictive accuracy and uncertainty quantification, providing deeper insights into the challenges of inference in this class of models.