期刊
EDUCATIONAL EVALUATION AND POLICY ANALYSIS
卷 23, 期 4, 页码 343-364出版社
SAGE PUBLICATIONS INC
DOI: 10.3102/01623737023004343
关键词
-
In this article we evaluate the peer review process used to fund Australian university, research across all disciplines. Peer reviews of research proposals (2,989 proposals, 6,233 external reviewers) submitted to the Australian Research Council (ARC) are related to characteristics of the researchers and of external reviewers. The reliability, of the peer reviews was disappointingly low (interrater agreement of .53 for researcher ratings based on an average of 4.3 external reviewers per proposal), The gender and age of a researcher and the number of researchers on a research team did not affect the probability that funding would be granted, but professors were more likely to be funded than nonprofessors. Australian external reviewers gave lower ratings than did non-Australian reviewers, particularly those from North America. The number of external reviewers for each proposal and the number of proposals assessed by each external reviewer had small negative effects on ratings. Researcher-nominated external reviewers (those chosen by the authors of a research proposal) gave higher, less-reliable ratings than did panel-nominated external reviewers chosen by, the ARC To improve the reliability of peer reviews, we offer the following recommendations: (a) Researcher-nominated reviewers should not be used; (b) there should be more reviews per proposal: and (c) a smaller number of more highly selected reviewers should perform most of the reviews within each subdiscipline, thereby providing greater control over error associated with individual reviewers.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据