Asymptotic is Better than Bollen-Stine Bootstrapping to Assess Model Fit: The Effect of Model Size on the Chi-Square Statistic

Previous research on bootstrapped p-values for the chi-square test of model fit has been limited to small models (around 10 variables), revealing that these p-values are accurate provided the sample size is not too small. For small sample sizes (N < 100), usual p-values, obtained using asymptotic methods, are more accurate. However, as the number of variables increases asymptotic p-values incorrectly suggest that models fit poorly. We investigate whether Bollen-Stine (1992) bootstrapped p-values can overcome this problem using normal and non-normal data. We found that as model size increases, Bollen-Stine bootstrapped p-values become too conservative and less accurate than asymptotic p-values obtained using robust methods (i.e., mean and variance corrections). Bollen-Stine p-values cannot be recommended to assess model fit.

Automated summary

Previous research has shown that bootstrapped p-values for the chi-square test of model fit are accurate for small models but not as accurate for small sample sizes. As the number of variables increases, bootstrapped p-values may become too conservative and less accurate. Therefore, they may not be recommended for assessing model fit.