Limitations of the Sum-and-Alpha Approach to Measurement in Behavioral Research

Many behavioral researchers are interested in measuring constructs such as mood, affect, or cognition-all of which cannot be observed directly. Instead, researchers administer items from surveys, tests, or scales to indirectly measure aspects of the construct. Psychometrics is a branch of statistics dedicated to determining whether scores created from item responses are reasonably capturing the intended construct. However, several review papers have shown that behavioral research frequently does not engage with psychometrics and instead creates scores by assigning numerical values to item responses, summing item responses, and reporting reliability of summed scores without any assessment of whether scores are valid. Despite the popularity of this approach, it can limit the informativeness and generalizability of conclusions in behavioral research. The goal of this paper is to raise awareness of these issues with policymakers and other consumers of research and to encourage producers of research to consider readily available, more rigorous approaches.

Automated summary

This paper highlights the importance of psychometrics in behavioral research, as many studies overlook it and fail to assess the validity of scores. This can limit the informativeness and generalizability of conclusions. The aim of the paper is to raise awareness among policymakers and research consumers and encourage the use of more rigorous approaches.