4.5 Article

A new algorithm for modeling online search behavior and studying ranking reliability variations

Journal

APPLIED INTELLIGENCE
Volume 52, Issue 7, Pages 7529-7549

Publisher

SPRINGER
DOI: 10.1007/s10489-021-02856-8

Keywords

Online search algorithm; Information retrieval; Ranking reliability; Decision tree; Click-through rate

Funding

  1. Czech Science Foundation [GAR19-13946S]

Ask authors/readers for more resources

The study designed an information retrieval algorithm that mimics the behavior of decision-makers when evaluating alternatives displayed by an online search engine. Experimental results show that stability of click-through rates prevails among top-ranked alternatives in relatively reliable scenarios, but drops when initial reliability decreases.
We design an information retrieval algorithm that mimics the stochastic behavior of decision-makers (DMs) when evaluating the alternatives displayed by an online search engine. The algorithm consists of a decision tree that incorporates all the 1024 decision nodes that may arise from the information retrieval process of DMs. We calibrate the behavior of the algorithm to the one observed from online users and run several sets of 1,000,000 queries. Each query lets DMs decide which subset of the ten alternatives composing the initial page of results to click, allowing us to evaluate their behavior as ranking reliability is assumed to decrease when DMs decide not to click on an alternative. We compare the click-through rates (CTRs) obtained when modifying the degree of ranking reliability derived from the alternatives displayed on the first page of search results. We illustrate how the stability of the CTR prevails among the top-ranked alternatives within relatively reliable scenarios while it drops when imposing large initial decrements in reliability. The resulting consequences regarding the importance of relative ranking positions are analyzed, the top three alternatives exhibiting a generally contained decrease in their CTRs that contrasts with the cumulative pattern arising from the fourth position onwards.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.5
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

No Data Available
No Data Available