Fit indices versus test statistics

Model evaluation is one of the most important aspects of structural equation modeling (SEM). Many model fit indices have been developed. It is not an exaggeration to say that nearly every publication using the SEM methodology has reported at least one fit index. Most fit indices are defined through test statistics. Studies and interpretation of fit indices commonly assume that the test statistics follow either a central chi-square distribution or a noncentral chi-square distribution. Because few statistics in practice follow a chi-square distribution, we study properties of the commonly used fit indices when dropping the chi-square distribution assumptions. The study identifies two sensible statistics for evaluating fit indices involving degrees of freedom. We also propose linearly approximating the distribution of a fit index/statistic by a known distribution or the distribution of the same fit index/statistic under a set of different conditions. The conditions include the sample size, the distribution of the data as well as the base-statistic. Results indicate that, for commonly used fit indices evaluated at sensible statistics, both the slope and the intercept in the linear relationship change substantially when conditions change. A fit index that changes the least might be due to an artificial factor. Thus, the value of a fit index is not just a measure of model fit but also of other uncontrollable factors. A discussion with conclusions is given on how to properly use fit indices.