P-Hacking,P-Curves, and thePSM-Performance Relationship: Is There Evidential Value?

Recent developments in the social sciences have demonstrated that we cannot uncritically aggregate the published research on a particular effect to conclude about its presence or absence. Instead, questionable research practices such asp-hacking (conducting additional analyses or collecting new data to obtain significant results) and selective publication of significant results can produce a body of published research that misleads readers even if it contains many significant results. It is, therefore, necessary to assess the evidential value of the research on a certain effect; that is, one must rule out that it is the result of questionable research practices. We introduce the p-curve method to public administration research and apply it to the research on the relationship between public service motivation (PSM) and individual performance, to demonstrate how the evidential value of a body of published research can be assessed. We find that this particular literature contains evidential value.

Automated summary

Recent developments in the social sciences have highlighted the need to critically assess published research in order to rule out the effects of questionable research practices that may lead to misleading conclusions. The introduction of the p-curve method offers a way to evaluate the evidential value of a body of research, as demonstrated in the analysis of the relationship between public service motivation (PSM) and individual performance.