Detecting Careless Respondents in Multidimensional Forced Choice Data: An Application of lz Person-Fit Statistic to the TIRT Model

There has been growing interest in the use of multidimensional forced-choice (MFC) measures in industrial and organizational psychology. However, despite the growing popularity, there has been a lack of research on how to address the problem of careless responses and improve the quality of MFC data. This study aims to address this gap by discussing the issue of careless responses in MFC data, proposing a method for detecting careless respondents using the l(z) person-fit statistic in conjunction with the Thurstonian item response theory (TIRT; Brown & Maydeu-Olivares, 2011) model, and examining the performance of this method on pair and triplet MFC measures and its impact on psychometric properties. To this end, we conducted empirical and simulation studies under various experimental conditions. We also provided an R tutorial for applying the l(z) person-fit statistic to both pair and triplet MFC measures using the TIRT model. The results showed that our proposed method can effectively detect careless respondents without a significant risk of misclassification. The implications of the findings and practical guidelines were discussed.

Automated summary

This study addresses the problem of careless responses in multidimensional forced-choice (MFC) data by proposing a method for detecting careless respondents and examining its impact on different MFC measures and psychometric properties. Empirical and simulation studies show that the proposed method effectively detects careless respondents without a significant risk of misclassification.