Measurement Invariance in International Surveys: Categorical Indicators and Fit Measure Performance

In spite of the challenges inherent in making dozens of comparisons across heterogeneous populations, a relatively recent interest in scale-score equivalence for non-achievement measures in an international context has emerged. Until recently, operational procedures for establishing measurement invariance using multiple-groups analyses were typically supported with research that was limited in scope to few groups and relatively small sample sizes. Recent research that examined situations more representative of international surveys recommended some revisions to typically used fit measures. The current study extends this research and evaluates the performance of several fit measures when data are assumed to have an ordered categorical, rather than the typically assumed continuous, scale. Using a simulation study based on empirical results, findings indicated that classic measures and associated criteria were either unsuitable in a large-group and varied sample-size context or should be adjusted, particularly when observed variables were not normally distributed. We provide specific recommendations for revising currently used criteria for evaluating overall and relative fit based on the chi-square test, root mean-squared error of approximation, and comparative fit index (CFI).