Open Science Standards at Journals that Inform Evidence-Based Policy

Evidence-based policy uses intervention research to inform consequential decisions about resource allocation. Research findings are often published in peer-reviewed journals. Because detrimental research practices associated with closed science are common, journal articles report more false-positives and exaggerated effect sizes than would be desirable. Journal implementation of standards that promote open science-such as the transparency and openness promotion (TOP) guidelines-could reduce detrimental research practices and improve the trustworthiness of research evidence on intervention effectiveness. We evaluated TOP implementation at 339 peer-reviewed journals that have been used to identify evidence-based interventions for policymaking and programmatic decisions. Each of ten open science standards in TOP was not implemented in most journals' policies (instructions to authors), procedures (manuscript submission systems), or practices (published articles). Journals implementing at least one standard typically encouraged, but did not require, an open science practice. We discuss why and how journals could improve implementation of open science standards to safeguard evidence-based policy.

Automated summary

Evidence-based policy relies on intervention research for significant decision-making on resource allocation. However, closed science practices in research publication contribute to false positives and exaggerated results. Implementing open science standards, such as the Transparency and Openness Promotion (TOP) guidelines, in journals can reduce detrimental practices and enhance the reliability of research evidence. Our evaluation of 339 peer-reviewed journals revealed a lack of implementation of TOP standards across policies, procedures, and practices. We discuss the need for and ways to improve the implementation of open science standards in journals to safeguard evidence-based policy.