Citations versus expert opinions: citation analysis of featured reviews of the American Mathematical Society

Peer review and citation metrics are two means of gauging the value of scientific research, but the lack of publicly available peer review data makes the comparison of these methods difficult. Mathematics can serve as a useful laboratory for considering these questions because as an exact science, there is a narrow range of reasons for citations. In mathematics, virtually all published articles are post-publication reviewed by mathematicians in Mathematical Reviews (MathSciNet) and so the data set was essentially the Web of Science mathematics publications from 1993 to 2004. For a decade, especially important articles were singled out in Mathematical Reviews for featured reviews. In this study, we analyze the bibliometrics of elite articles selected by peer review and by citation count. We conclude that the two notions of significance described by being a featured review article and being highly cited are distinct. This indicates that peer review and citation counts give largely independent determinations of highly distinguished articles. We also consider whether hiring patterns of subfields and mathematicians' interest in subfields reflect subfields of featured review or highly cited articles. We re-examine data from two earlier studies in light of our methods for implications on the peer review/citation count relationship to a diversity of disciplines.

Automated summary

The study analyzed the bibliometrics of elite articles selected by peer review and by citation count, highlighting the largely independent determinations of highly distinguished articles given by peer review and citation counts. Mathematics serving as a laboratory demonstrated that the two notions of significance are distinct, indicating significant differences in the evaluations of highly distinguished articles using peer review and citation count methods.