Assessing Approximate Fit in Categorical Data Analysis

A family of Root Mean Square Error of Approximation (RMSEA) statistics is proposed for assessing the goodness of approximation in discrete multivariate analysis with applications to item response theory (IRT) models. The family includes RMSEAs to assess the approximation up to any level of association of the discrete variables. Two members of this family are RMSEA(2), which uses up to bivariate moments, and the full information RMSEA(n). The RMSEA(2) is estimated using the M-2 statistic of Maydeu-Olivares and Joe (2005, 2006), whereas for maximum likelihood estimation, RMSEA(n) is estimated using Pearson's X-2 statistic. Using IRT models, we provide cutoff criteria of adequate, good, and excellent fit using the RMSEA(2). When the data are ordinal, we find a strong linear relationship between the RMSEA(2) and the Standardized Root Mean Squared Residual goodness-of-fit index. We are unable to offer cutoff criteria for the RMSEA(n) as its population values decrease as the number of variables and categories increase.