Intuitive joint priors for Bayesian linear multilevel models: The R2D2M2 prior

The training of high-dimensional regression models on comparably sparse data is an important yet complicated topic, especially when there are many more model parameters than observations in the data. From a Bayesian perspective, inference in such cases can be achieved with the help of shrinkage prior distributions, at least for generalized linear models. However, real-world data usually possess multilevel structures, such as repeated measurements or natural groupings of individuals, which existing shrinkage priors are not built to deal with. We generalize and extend one of these priors, the R2D2 prior by Zhang et al. (2020), to linear multilevel models leading to what we call the R2D2M2 prior. The proposed prior enables both local and global shrinkage of the model parameters. It comes with interpretable hyperparameters, which we show to be intrinsically related to vital properties of the prior, such as rates of concentration around the origin, tail behavior, and amount of shrinkage the prior exerts. We offer guidelines on how to select the prior's hyperparameters by deriving shrinkage factors and measuring the effective number of non-zero model coefficients. Hence, the user can readily evaluate and interpret the amount of shrinkage implied by a specific choice of hyperparameters. Finally, we perform extensive experiments on simulated and real data, showing that our inference procedure for the prior is well calibrated, has desirable global and local regularization properties and enables the reliable and interpretable estimation of much more complex Bayesian multilevel models than was previously possible.

Automated summary

Training high-dimensional regression models on sparse data with more parameters than observations is complex. Bayesian inference can be achieved using shrinkage priors, but existing ones do not handle multilevel structures. We propose the R2D2M2 prior, an extension of the R2D2 prior, for linear multilevel models. The proposed prior enables both local and global shrinkage, has interpretable hyperparameters, and allows evaluation and interpretation of shrinkage. Extensive experiments show its effectiveness for estimating complex Bayesian multilevel models.