Weighted Bayesian bootstrap for scalable posterior distributions

We introduce and develop a weighted Bayesian bootstrap (WBB) for machine learning and statistics. WBB provides uncertainty quantification by sampling from a high dimensional posterior distribution. WBB is computationally fast and scalable using only off-the-shelf optimization software. First-order asymptotic analysis provides a theoretical justification under suitable regularity conditions on the statistical model. We illustrate the proposed methodology in regularized regression, trend filtering and deep learning and conclude with directions for future research.

Automated summary

The Weighted Bayesian Bootstrap (WBB) method is introduced and developed for machine learning and statistics, providing uncertainty quantification by sampling from a high dimensional posterior distribution. The WBB is computationally fast and scalable using off-the-shelf optimization software, and has theoretical justification under suitable regularity conditions on the statistical model. Applications in regularized regression, trend filtering, and deep learning are demonstrated, with suggestions for future research directions.