Variational Bayesian inference with complex geostatistical priors using inverse autoregressive flows

We combine inverse autoregressive flows (IAF) and variational Bayesian inference (variational Bayes) in the context of geophysical inversion parameterized with deep generative models encoding complex priors. Variational Bayes approximates the unnormalized posterior distribution parametrically within a given family of distributions by solving an optimization problem. Although prone to bias if the chosen family of distributions is too limited, it provides a computationally-efficient approach that scales well to high-dimensional inverse problems. To enhance the expressiveness of the variational distribution, we explore its combination with IAFs that allow samples from a simple base distribution to be pushed forward through a series of invertible transformations onto an approximate posterior. The IAF is learned by maximizing the lower bound of the evidence (marginal likelihood), which is equivalent to minimizing the Kullback-Leibler divergence between the approximation and the target posterior distribution. In our examples, we use either a deep generative adversar-ial network (GAN) or a variational autoencoder (VAE) to parameterize complex geostatistical priors. Although previous attempts to perform Gauss-Newton inversion in combination with GANs of the same architecture were proven unsuccessful, the trained IAF provides a good reconstruction of channelized subsurface models for both GAN-and VAE-based inversions using synthetic crosshole ground-penetrating-radar data. For the considered examples, the computational cost of our approach is seven times lower than for Markov chain Monte Carlo (MCMC) inversion. Furthermore, the VAE-based approximations in the latent space are in good agreement. The VAE-based inversion requires only one sample to estimate gradients with respect to the IAF parameters at each iteration, while the GAN-based inversions need more samples and the corresponding posterior approximation is less accurate.

Automated summary

In this study, we combine inverse autoregressive flows (IAF) with variational Bayesian inference (variational Bayes) for geophysical inversion using deep generative models with complex priors. Variational Bayes parameterizes the unnormalized posterior distribution within a given family of distributions through optimization. To enhance the expressiveness of the variational distribution, we explore its combination with IAFs, which transform samples from a base distribution onto an approximate posterior. The trained IAF provides a good reconstruction of channelized subsurface models for GAN and VAE-based inversions using synthetic crosshole ground-penetrating-radar data.