Using full models, stepwise regression and model selection in ecological data sets: Monte Carlo simulations

Most ecological studies with multiple independent variables use null hypothesis testing with full or stepwise models, or AICc-based model selection, but these approaches have not yet been compared using simulated data with known effect sizes. We compared these using ecologically relevant sample sizes, effect sizes, predictor numbers, collinearity and different degrees of explorative setups. Sample size and collinearity governed parameter identification success and parameter estimation accuracy, while the effect of the statistical modeling approach was comparatively smaller. Stepwise regression increased false detection rate compared with full models in settings where this error rate was overall low, but generally reduced the high detection failure rate in small samples. When reintroducing removed predictors to the final model, stepwise regression often improved the accuracy of point estimates relative to full models. The performance of AICc model selection and model averaging depended on the exact method, and did not differ overall from null hypothesis testing approaches.