Biomedical databases and the journal impact factor

Background: Citation frequencies of medical journals are measured by the Journal Impact Factors (IF) published annually in the Journal Citation Report (JCR). Conclusions drawn from IF concerning the distribution of single journal articles are used worldwide for academic evaluation purposes. Because of this importance, IF are widely and controversially discussed, also regarding their derivation from a limited pool of databases (Science Citation Index, SCI). Objective: To determine the comprehensiveness of IF's data basis by testing (i) SCI's sources, (ii) SCI's output. Are the IF sufficient for an objective evaluation of medical journals. Methods: Comparative searches in 38 databases and their combinations (SCI; MEDLINE [ME]; EMBASE [EM]; BIOSIS Previews [BA] and other relevant systems). Results: (i) Journals with higher IF (greater than or equal to 1)are almost completely retrievable in SCI (98%), but only approx. 60% in ME, EM, BA. (ii) Reverse: three samples of mainly German-language journals frequently indexed in SCI were represented in JCR by 90%; but only 23,5-57% of sample periodicals had an IF when indexed in ME, EM and BA, but not in SCI. (iii) Compiled average search results in the mast productive databases in 18 biomedical queries, when titles were searched: SCI=34%, ME=27%, EM=33%, BA=25%; and, when combined: SCI + ME=44%, SCI i ME i EM=55%, SCI i ME + EM i BA=65%, compared to the results in a 38-databases cluster. Costs increase in the order ME < EM < SCI < BA < Derwent, GAS. Conclusions: (i) The citation analyses presented in JCRs appear limited especially regarding German-language biomedical journals. Evaluation of publications based on IF therefore should be complemented by corrective measures. (ii) Single-database searches, including SCI, at best render orientating results; database combinations are recommended when higher completeness is required.