Semi-automated Rasch analysis using in-plus-out-of-questionnaire log likelihood

Rasch analysis is a popular statistical tool for developing and validating instruments that aim to measure human performance, attitudes and perceptions. Despite the availability of various software packages, constructing a good instrument based on Rasch analysis is still considered to be a complex, labour-intensive task, requiring human expertise and rather subjective judgements along the way. In this paper we propose a semi-automated method for Rasch analysis based on first principles that reduces the need for human input. To this end, we introduce a novel criterion, called in-plus-out-of-questionnaire log likelihood (IPOQ-LL). On artificial data sets, we confirm that optimization of IPOQ-LL leads to the desired behaviour in the case of multi-dimensional and inhomogeneous surveys. On three publicly available real-world data sets, our method leads to instruments that are, for all practical purposes, indistinguishable from those obtained by Rasch analysis experts through a manual procedure.

Automated summary

In this paper, a semi-automated method for Rasch analysis based on first principles is proposed, which reduces the need for human input by introducing a novel criterion called IPOQ-LL. The optimization of IPOQ-LL is confirmed to lead to the desired behavior in multi-dimensional and inhomogeneous surveys. Additionally, this method is shown to produce instruments that are practically indistinguishable from those obtained by experts through manual procedures on real-world data sets.