Structured Shrinkage Priors

In many regression settings the unknown coefficients may have some known structure, for instance they may be ordered in space or correspond to a vectorized matrix or tensor. At the same time, the unknown coefficients may be sparse, with many nearly or exactly equal to zero. However, many commonly used priors and corresponding penalties for coefficients do not encourage simultaneously structured and sparse estimates. In this article we develop structured shrinkage priors that generalize multivariate normal, Laplace, exponential power and normal-gamma priors. These priors allow the regression coefficients to be correlated a priori without sacrificing elementwise sparsity or shrinkage. The primary challenges in working with these structured shrinkage priors are computational, as the corresponding penalties are intractable integrals and the full conditional distributions that are needed to approximate the posterior mode or simulate from the posterior distribution may be nonstandard. We overcome these issues using a flexible elliptical slice sampling procedure, and demonstrate that these priors can be used to introduce structure while preserving sparsity. Supplementary materials for this article are available online.

Automated summary

In many regression settings, the unknown coefficients may have known structure and be sparse. However, commonly used priors and penalties do not encourage both structured and sparse estimates. This article develops structured shrinkage priors that allow for correlated coefficients while maintaining sparsity, and presents a computational approach to overcome the challenges associated with these priors. The results demonstrate the effectiveness of these priors in introducing structure while preserving sparsity.