Automatic Generation of Initial Reading Lists: Requirements and Solutions

Researchers who start delving into a new research area often resort to reading lists of scientific articles in order to familiarize themselves with the existing literature. Generally, these reading lists are manually built and recommended by experts. In this paper we tackle the problem of automatically generating these lists. We start by stating five requirements for a comprehensive initial reading list, four of which were previously proposed and one is a contribution of ours. We then assess the extent to which these requirements are redundant or complementary. By performing a correlation analysis on a large dataset, we find that the five requirements are indeed mostly conflicting, which suggests that simultaneously meeting all of them is a difficult task. We then perform an extensive set of experiments, comparing twenty-five different approaches to automatically generate initial reading lists, most of which are new proposals of ours which exploit learning to rank (L2R) and aggregation methods to combine multiple pieces of evidence and objectives. Our experimental results indicate that, though no method outperforms the others in all five requirements, our new L2R and aggregation methods significantly outperform the state-of-the-art when jointly considering all requirements. Moreover, we identify a subset of six new techniques which offer the best tradeoff (in a Pareto-efficient sense) across all five requirements.