User-defined relevance criteria in web searching

Purpose - The purpose of this paper is to specify user-defined relevance criteria by which people Accepted 30 November 2005 select hyperlinks and pages in web searching. Design/methodology/approach - A quantitative and qualitative analysis was undertaken of talking aloud data from nine web searches conducted about self-generated topics. Findings - Altogether 18 different criteria for selecting hyperlinks and web pages were found. The selection is constituted, by two, intertwined processes: the relevance judgment of hyperlinks, and web pages by user-defined criteria, and decision-making concerning the acceptance or rejection of hyperlinks and web pages. The study focuses on the former process. Of the individual criteria, specificity, topicality, familiarity, and variety were used most frequently in relevance judgments. The study shows that despite the high number of individual criteria used in the judgments, a few criteria such as specificity and topicality tend to dominate. Searchers were less critical in the judgment of hyperlinks than deciding whether the activated web pages should be consulted in more detail. Research limitations/implications - The study is exploratory, drawing on a relatively low number of case searches. Originality/value - The paper gives a detailed picture of the criteria used in the relevance judgments of hyperlinks and web pages. The study also discusses the specific nature of criteria used in web searching, as compared to those used in traditional online searching environments.