Assessing the Accuracy of Errors of Measurement. Implications for Assessing Reliable Change in Clinical settings

Item response theory (IRT) models are non-linear latent variable models for discrete measures, whereas factor analysis (FA) is a latent variable model for continuous measures. In FA, the standard error (SE) of individuals' scores is common for all individuals. In IRT, the SE depends on the individual's score, and the SE function is to be provided. The empirical standard deviation of the scores across discrete ranges should also be computed to inform the extent to which IRT SEs overestimate or underestimate the variability of the scores. Within the target range of scores the test was designed to measure, one should expect IRT SEs to be smaller and more precise than FA SEs, and therefore preferable to assess clinical change. Outside the target range, IRT SEs may be too large and more imprecise than FA SEs, and FA more precise to assess change. As a result, whether FA or IRT characterize reliable change more accurately in a sample will depend on the proportion of individuals within or outside the IRT target score range. An application is provided to illustrate these concepts.

Automated summary

Item response theory (IRT) and factor analysis (FA) are models used for measuring different variables, with differences in calculating standard errors. In the target range, IRT is typically more precise than FA, while FA is more precise outside the target range.