Providing a Lower-Bound Estimate for Psychology's Crud Factor: The Case of Aggression

When conducting research on large data sets, statistically significant findings having only trivial interpretive meaning may appear. Little consensus exists whether such small effects can be meaningfully interpreted. The current analysis examines the possibility that trivial effects may emerge in large datasets, but that some such effects may lack interpretive value. When such results match an investigator's hypothesis, they may be over-interpreted. The current study examines this issue as related to aggression research in two large samples. Specifically, in the first study, the National Longitudinal Study of Adolescent to Adult Health (AddHeath) dataset was used. Fifteen variables with little theoretical relevance to aggression were selected, then correlated with self-reported delinquency. For the second study, the Understanding Society database was used. As with Study 1, 14 nonsensical variables were correlated with conduct problems. Many variables achieved statistical significance and some effect sizes approached or exceeded r = .10, despite little theoretical relevance between the variables. It is recommended that effect sizes below r = .10 should not be interpreted as hypothesis supportive.

Automated summary

This study found that statistically significant but trivial effects may emerge in large datasets, but these effects may lack interpretive value, cautioning against over-interpretation.Variables with little theoretical relevance achieved significance, suggesting that effect sizes below r = .10 should not be interpreted as hypothesis supportive.