The overreliance on statistical goodness-of-fit and under-reliance on model validation in discrete choice models: A review of validation practices in the transportation academic literature

An examination of model validation practices in the peer-reviewed transportation literature published between 2014 and 2018 reveals that 92% of studies reported goodness-of-fit statistics, and 64.6% reported some sort of policy-relevant inference analysis. However, only 18.1% reported validation performance measures, out of which 78% (14.2% of all studies) consisted of internal validation and 22% (4% of all studies) consisted of external validation. The proposition put forward in this paper is that the reliance on goodness-of-fit measures rather than validation performance is unwise, especially given the dependence of the transportation research field on observational (non-experimental) studies. Model validation should be a non-negotiable part of presenting a model for peer-review in academic journals. For that purpose, we propose a simple heuristic to select validation methods given the resources available to the researcher.

Automated summary

An examination of transportation literature published between 2014 and 2018 reveals a lack of emphasis on validation performance measures, with most studies focusing on goodness-of-fit statistics and policy-relevant inference analysis. The paper argues that model validation should be a non-negotiable part of presenting a model for peer-review in academic journals, and proposes a simple heuristic for selecting validation methods based on available resources.