An Excess of Positive Results: Comparing the Standard Psychology Literature With Registered Reports

Selectively publishing results that support the tested hypotheses (positive results) distorts the available evidence for scientific claims. For the past decade, psychological scientists have been increasingly concerned about the degree of such distortion in their literature. A new publication format has been developed to prevent selective reporting: In Registered Reports (RRs), peer review and the decision to publish take place before results are known. We compared the results in published RRs (N = 71 as of November 2018) with a random sample of hypothesis-testing studies from the standard literature (N = 152) in psychology. Analyzing the first hypothesis of each article, we found 96% positive results in standard reports but only 44% positive results in RRs. We discuss possible explanations for this large difference and suggest that a plausible factor is the reduction of publication bias and/or Type I error inflation in the RR literature.

Automated summary

Selective publishing of positive results may distort the evidence for scientific claims. A comparison between Registered Reports (RRs) and standard literature in psychology reveals a significant difference in the proportion of positive results, suggesting a reduction in publication bias and/or Type I error inflation in RR literature.