Expansion of scientific journal categories using reference analysis: How can it be done and does it make a difference?

This paper explores a methodology for delimitating scientific subfields by combining the use of (specialist) journal categories from Thomson Scientific's Web of Science (WoS) and reference analysis. In a first step it selects all articles in journals included in a particular WoS journal category covering a subfield. These journals are labelled as a subfield's specialist journals. In a second step, this set of papers is expanded with papers published in other, additional journals and citing a subfield's specialist journals with a frequency exceeding a certain citation threshold. Data are presented for two medical subfields: Oncology and Cardiac & Cardiovascular System. A validation based on findings from earlier studies, from an analysis of MESH descriptors from MEDLINE, and on expert opinion provides evidence that the proposed methodology has a high precision, and that expansion substantially enhanced the recall, not merely in terms of the number of retrieved papers, but also in terms of the number of research topics covered. The paper also examines how a bibliometric ranking of countries and universities based on the citation impact of their papers published in a subfield's specialist journals compares to that of a ranking based on the impact of their articles in additional journals. Rather weak correlations especially obtained at the level of universities underline the conclusion from earlier studies that an assessment of research groups or universities in a scientific subfield that takes into account solely papers published in a subfield's specialist journals is unsatisfactory.