Multilevel Mediation With Small Samples: A Cautionary Note on the Multilevel Structural Equation Modeling Framework

Multilevel structural equation modeling (ML-SEM) for multilevel mediation is noted for its flexibility over a system of multilevel models (MLMs). Sample size requirements are an overlooked limitation of ML-SEM (100 clusters is recommended). We find that 89% of ML-SEM studies have fewer than 100 clusters and the median number is 44. Furthermore, 75% of ML-SEM studies implement 2-1-1 or 1-1-1 models, which can be equivalently fit with MLMs. MLMs theoretically have lower sample size requirements, although studies have yet to assess small sample performance for multilevel mediation. We conduct a simulation to address this pervasive problem. We find that MLMs have more desirable small sample performance and can be trustworthy with 10 clusters. Importantly, many studies lack the sample size and model complexity to necessitate ML-SEM. Although ML-SEM is undeniably more flexible and uniquely positioned for difficult problems, small samples often can be more effectively and simply addressed with MLMs.