A Core-Item Reviewer Evaluation (CoRE) System for Manuscript Peer Review

Manuscript peer review is essential for ensuring accountability to all involved in the publication process, including authors, journals, and readers. Lack of consensus regarding what constitutes an accountable manuscript peer review process has resulted in varying practices from one journal to the next. Currently, reviewers are asked to make global judgments about various aspects of a paper for review irrespective of whether guided by a review checklist or not, and several studies have documented gross disagreement between reviewers of the same manuscript. We have previously proposed that the solution may be to direct reviewers to concrete items that do not require global judgments but rather provide a specific choice, along with referee justification for such choices. This study evaluated use of such a system via an international survey of health care professionals who had recently reviewed a health care--related manuscript. Results suggest that use of such a peer review system by reviewers does indeed improve interreviewer agreement, and thus, has the potential to support more consistent and effective peer review, if introduced into journal processes for peer review.