Are Small Effects the Indispensable Foundation for a Cumulative Psychological Science? A Reply to Gotz et al. (2022)

In the January 2022 issue of Perspectives, Gotz et al. argued that small effects are the indispensable foundation for a cumulative psychological science. They supported their argument by claiming that (a) psychology, like genetics, consists of complex phenomena explained by additive small effects; (b) psychological-research culture rewards large effects, which means small effects are being ignored; and (c) small effects become meaningful at scale and over time. We rebut these claims with three objections: First, the analogy between genetics and psychology is misleading; second, p values are the main currency for publication in psychology, meaning that any biases in the literature are (currently) caused by pressure to publish statistically significant results and not large effects; and third, claims regarding small effects as important and consequential must be supported by empirical evidence or, at least, a falsifiable line of reasoning. If accepted uncritically, we believe the arguments of Gotz et al. could be used as a blanket justification for the importance of any and all small effects, thereby undermining best practices in effect-size interpretation. We end with guidance on evaluating effect sizes in relative, not absolute, terms.

Automated summary

Gotz et al. argue that small effects are crucial for cumulative psychological science, but we raise objections based on the misleading analogy between genetics and psychology, the reliance on p values in psychology publications, and the necessity for empirical evidence or falsifiable reasoning when claiming the importance of small effects.