Multilevel mediation analysis in R: A comparison of bootstrap and Bayesian approaches

Mediation analysis in repeated measures studies can shed light on the mechanisms through which experimental manipulations change the outcome variable. However, the literature on interval estimation for the indirect effect in the 1-1-1 single mediator model is sparse. Most simulation studies to date evaluating mediation analysis in multilevel data considered scenarios that do not match the expected numbers of level 1 and level 2 units typically encountered in experimental studies, and no study to date has compared resampling and Bayesian methods for constructing intervals for the indirect effect in this context. We conducted a simulation study to compare statistical properties of interval estimates of the indirect effect obtained using four bootstrap and two Bayesian methods in the 1-1-1 mediation model with and without random effects. Bayesian credibility intervals had coverage closest to the nominal value and no instances of excessive Type I error rates, but lower power than resampling methods. Findings indicated that the pattern of performance for resampling methods often depended on the presence of random effects. We provide suggestions for selecting an interval estimator for the indirect effect depending on the most important statistical property for a given study, as well as code in R for implementing all methods evaluated in the simulation study. Findings and code from this project will hopefully support the use of mediation analysis in experimental research with repeated measures.

Automated summary

Mediation analysis helps understand how experimental manipulations change outcome variables. However, research on interval estimation for indirect effect in the 1-1-1 single mediator model is limited. Simulation studies on mediation analysis in multilevel data often do not match the typical scenarios encountered in experimental studies, and no study has compared resampling and Bayesian methods for constructing intervals in this context. Our simulation study compared the properties of interval estimates using bootstrap and Bayesian methods in the 1-1-1 mediation model. Bayesian credibility intervals performed well, but had lower power compared to resampling methods. The findings provide suggestions for selecting interval estimators for indirect effect based on the most important statistical property for a given study, and include R code for implementing all methods.