Correcting for Extreme Response Style: Model Choice Matters

Extreme response style (ERS), the tendency of participants to select extreme item categories regardless of the item content, has frequently been found to decrease the validity of Likert-type questionnaire results. For this reason, various item response theory (IRT) models have been proposed to model ERS and correct for it. Comparisons of these models are however rare in the literature, especially in the context of cross-cultural comparisons, where ERS is even more relevant due to cultural differences between groups. To remedy this issue, the current article examines two frequently used IRT models that can be estimated using standard software: a multidimensional nominal response model (MNRM) and a IRTree model. Studying conceptual differences between these models reveals that they differ substantially in their conceptualization of ERS. These differences result in different category probabilities between the models. To evaluate the impact of these differences in a multigroup context, a simulation study is conducted. Our results show that when the groups differ in their average ERS, the IRTree model and MNRM can drastically differ in their conclusions about the size and presence of differences in the substantive trait between these groups. An empirical example is given and implications for the future use of both models and the conceptualization of ERS are discussed.

Automated summary

This article examines two frequently used item response theory (IRT) models, the multidimensional nominal response model (MNRM) and the IRTree model. The study reveals conceptual differences between these models, which result in different conclusions about the size and presence of differences in substantive trait between groups. A simulation study shows that the IRTree model and MNRM can drastically differ in their conclusions when groups differ in their average extreme response style (ERS). An empirical example is provided and implications for the future use of both models and the conceptualization of ERS are discussed.