Pervasiveness of effects in sample-based experimental psychology: A Re-examination of replication data from nine famous psychology experiments

Data previously reported from successful replications of 9 famous experiments (Zwaan et al., 2018) were re-analysed. Rather than the null hypothesis significance tests performed in the original study, this study per-formed a pervasiveness analysis, which examined the number of people in the original samples that demon-strated the effect each experiment is renowned for. Seven of the 9 experiments were shown to exceed the pervasiveness threshold adopted in this study. That is, in each of these 7 experiments, over 80% of participants demonstrated the target effect, indicating support for Zwaan et al.'s claim that these effects are robust. The remaining 2 experiments (Repetition Priming and Shape Simulation) did not meet this pervasiveness criterion, casting doubt on the reproducibility of these effects. The pervasiveness analysis was demonstrated to be a useful adjunct to traditional forms of analysis because it provides information directly relevant to claims about the reproducibility of psychological effects.

Automated summary

This study re-analyzed data from 9 famous experiments that had been successfully replicated, showing that 7 of these experiments demonstrated a pervasiveness of over 80%, supporting the robustness of these effects. However, the remaining 2 experiments did not meet the pervasiveness criterion, raising doubts about their reproducibility. The pervasiveness analysis proved to be a useful adjunct to traditional forms of analysis, providing directly relevant information regarding the reproducibility of psychological effects.