Multilevel Modeling of Single-Case Data: A Comparison of Maximum Likelihood and Bayesian Estimation

The focus of this article is to describe Bayesian estimation, including construction of prior distributions, and to compare parameter recovery under the Bayesian framework (using weakly informative priors) and the maximum likelihood (ML) framework in the context of multilevel modeling of single-case experimental data. Bayesian estimation results were found similar to ML estimation results in terms of the treatment effect estimates, regardless of the functional form and degree of information included in the prior specification in the Bayesian framework. In terms of the variance component estimates, both the ML and Bayesian estimation procedures result in biased and less precise variance estimates when the number of participants is small (i.e., 3). By increasing the number of participants to 5 or 7, the relative bias is close to 5% and more precise estimates are obtained for all approaches, except for the inverse-Wishart prior using the identity matrix. When a more informative prior was added, more precise estimates for the fixed effects and random effects were obtained, even when only 3 participants were included. Translational Abstract Variance component estimates in contexts of 2-level modeling of single-case experimental data tend to be biased and imprecisely estimated. In this study, Bayesian estimation is proposed as an alternative to a maximum likelihood estimation procedure. Prior selection methods and prior construction for the variance components in the Bayesian framework are described. The purpose is to compare parameter recovery of maximum likelihood estimation and Bayesian estimation using Monte Carlo simulation methods. Both maximum likelihood and Bayesian estimation procedures result in biased and less precise variance estimates when the number of participants is small (i.e., 3). By increasing the number of participants to 5 or 7, unbiased and more precise estimates are obtained for all priors, except for the inverse-Wishart prior using the identity matrix. When a more informative prior is added, more precise estimates were obtained, even when only 3 participants were included. An empirical example is given to illustrate the Bayesian approach. We recommend that applied researchers use a variety of different prior distributions (following the proposed prior selection and constriction) and discuss to what extent and in what sense the results depend on the prior used.