Journal
RESEARCH EVALUATION
Volume 30, Issue 3, Pages 314-322Publisher
OXFORD UNIV PRESS
DOI: 10.1093/reseval/rvab005
Keywords
peer review; open peer review; scholarly communication; language use
Categories
Funding
- University of Wisconsin-Milwaukee Research Growth Initiative Grant
Ask authors/readers for more resources
This study examines the characteristics of review content and quantitative aspects of the review process in journals adopting open peer review. The findings suggest that the use of hedging terms in reviews is not influenced by whether reviewers are required to identify themselves, while there is a significant difference in the use of research-related terms. Additionally, the study reveals that having more reviewers or conducting more reviews does not necessarily lead to more impactful papers.
Journals that adopt open peer review (OPR), where review reports of published articles are publicly available, provide an opportunity to study both review content characteristics and quantitative aspects of the overall review process. This study investigates two areas relevant to the quality assessment of manuscript reviews. First, do journal policies for reviewers to identify themselves influence how reviewers evaluate the merits of a manuscript based on the relative frequency of hedging terms and research-related terms appearing in their reviews? Second, is there an association between the number of reviews/reviewers and the manuscript's research impact once published as measured by citations? We selected reviews for articles published in 17 OPR journals from 2017 to 2018 to examine the incidence of reviewers' uses of hedging terms and research-related terms. The results suggest that there was little difference in the relative use of hedging term usage regardless of whether reviewers were required to identify themselves or if this was optional, indicating that the use of hedging in review contents was not influenced by journal requirements for reviewers to identify themselves. There was a larger difference observed for research-related terminology. We compared the total number of reviews for a manuscript, rounds of revisions, and the number of reviewers with the number of Web of Science citations the article received since publication. The findings reveal that scrutiny by more reviewers or conducting more reviews or rounds of review do not result in more impactful papers for most of the journals studied. Implications for peer review practice are discussed.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available