Academic information retrieval using citation clusters: in-depth evaluation based on systematic reviews

The field of science mapping has shown the power of citation-based clusters for literature analysis, yet this technique has barely been used for information retrieval tasks. This work evaluates the performance of citation-based clusters for information retrieval tasks. We simulated a search process with a tree hierarchy of clusters and a cluster selection algorithm. We evaluated the task of finding the relevant documents for 25 systematic reviews. Our evaluation considered several trade-offs between recall and precision for the cluster selection. We also replicated the Boolean queries self-reported by the systematic reviews to serve as a reference. We found that citation-based clusters' search performance is highly variable and unpredictable, that the clusters work best for users that prefer recall over precision at a ratio between 2 and 8, and that the clusters are able to complement query-based search by finding additional relevant documents.

Automated summary

The field of science mapping has shown the potential of citation-based clusters in literature analysis, but their application in information retrieval tasks is limited. This study assessed the performance of citation-based clusters in information retrieval tasks. A simulated search process with a tree hierarchy of clusters and a cluster selection algorithm was used. The evaluation focused on finding relevant documents for 25 systematic reviews, considering trade-offs between recall and precision for cluster selection. Results showed that the search performance of citation-based clusters is highly variable and unpredictable. Clusters work best for users who prioritize recall over precision at a ratio between 2 and 8, and they can complement query-based search by finding additional relevant documents.