A Comparison of Several Approaches for Controlling Measurement Error in Small Samples

It is well known that methods that fail to account for measurement error in observed variables, such as regression and path analysis (PA), can result in poor estimates and incorrect inference. On the other hand, methods that fully account for measurement error, such as structural equation modeling with latent variables and multiple indicators, can produce highly variable estimates in small samples. This article advocates a family of intermediate models for small samples (N < 200), referred to as single indicator (SI) models. In these models, each latent variable has a single composite indicator, with its reliability fixed to a plausible value. A simulation study compared three versions of the SI method with PA and with a multiple-indicator structural equation model (SEM) in small samples (N = 30 to 200). Two of the SI models fixed the reliability of each construct to a value chosen a priori (either .7 or .8). The third SI model (referred to as SI alpha) estimated the reliability of each construct from the data via coefficient alpha. The results showed that PA and fixed-reliability SI methods that overestimated reliability slightly resulted in the most accurate estimates as well as in the highest power. Fixed-reliability SI methods also maintained good coverage and Type I error rates. The SI alpha and SEM methods had intermediate performance. In small samples, use of a fixed-reliability SI method is recommended. Translational Abstract Most psychological variables cannot be measured precisely, but instead are measured with error. It is well known that statistical methods that fail to account for this measurement error can result in biased estimates and incorrect conclusions drawn from data. The most common such method is ordinary regression. On the other hand, more sophisticated methods that fully account for measurement error, such as structural equation modeling, produce unbiased but often imprecise estimates, especially when the data set is small. This article advocates a family of intermediate statistical methods for small data sets, referred to as single indicator methods. To implement this method, each psychological latent variable is represented by a sum of all observed measures that reflect it, with the amount of measurement error fixed to a reasonable estimate. The article reports on the results of a simulation study that shows that the single indicator method is better than using either methods that do not account for measurement error at all (such as regression) or methods that account for it in an overly complicated way (such as structural equation modeling). In small samples, use of single indicators is strongly recommended.