The Consequences of Ignoring Multilevel Data Structures in Nonhierarchical Covariance Modeling

This study examined the effects of ignoring multilevel data structures in nonhierarchical covariance modeling using a Monte Carlo simulation. Multilevel sample data were generated with respect to 3 design factors: (a) intraclass correlation, (b) group and member configuration, and (c) the models that underlie the between-group and within-group variance components associated with multilevel data. Covariance models that ignored the multilevel structure were then fit to the data. Results indicated that when variables exhibit minimal levels of intraclass correlation, the chi-square model/data fit statistic, the parameter estimators, and the standard error estimators are relatively unbiased. However, as the level of intraclass correlation increases, the chi-square statistic, the parameters, and their standard errors all exhibit estimation problems. The specific group/member configurations as well as the underlying between-group and within-group model structures further exacerbate the estimation problems encountered in the nonhierarchical analysis of multilevel data.