Differences in review quality and recommendations for publication between peer reviewers suggested by authors or by editors

Context Many journals give authors who submit papers the opportunity to suggest reviewers. Use of these reviewers varies by journal and little is known about the quality of the reviews they produce. Objective To compare author- and editor-suggested reviewers to investigate differences in review quality and recommendations for publication. Design, Setting, and Participants Observational study of original research papers sent for external review at 10 biomedical journals. Editors were instructed to make decisions about their choice of reviewers in their usual manner. Journal administrators then requested additional reviews from the author's list of suggestions according to a strict protocol. Main Outcome Measure Review quality using the Review Quality Instrument and the proportion of reviewers recommending acceptance (including minor revision), revision, or rejection. Results There were 788 reviews for 329 manuscripts. Review quality (mean difference in Review Quality Instrument score, -0.05; P=.27) did not differ significantly between author- and editor-suggested reviewers. The author-suggested reviewers were more likely to recommend acceptance (odds ratio, 1.64; 95% confidence interval, 1.02-2.66) or revise (odds ratio, 2.66; 95% confidence interval, 1.43-4.97). This difference was larger in the open reviews of BMJ than among the blinded reviews of other journals for acceptance (P=.02). Where author- and editor-suggested reviewers differed in their recommendations, the final editorial decision to accept or reject a study was evenly balanced (50.9% of decisions consistent with the preferences of the author-suggested reviewers). Conclusions Author- and editor-suggested reviewers did not differ in the quality of their reviews, but author-suggested reviewers tended to make more favorable recommendations for publication. Editors can be confident that reviewers suggested by authors will complete adequate reviews of manuscripts, but should be cautious about relying on their recommendations for publication.