Journal
JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY
Volume 64, Issue 10, Pages 2024-2044Publisher
WILEY
DOI: 10.1002/asi.22908
Keywords
information retrieval; knowledge management
Funding
- IOP-MMI program of SenterNovem/The Dutch Ministry of Economic Affairs, as part of the A Propos project
- Radio Culture and Auditory Resources Infrastructure Project (LARM)
- Danish National Research Infrastructures Program [09-067292]
- European Union's ICT Policy Support Programme as part of the Competitiveness and Innovation Framework Programme, CIP ICT-PSP [250430]
- European Community's Seventh Framework Programme (FP7) [258191, 288024]
- Netherlands Organisation for Scientific Research (NWO) [612.061.-814, 612.061.815, 640.004.802, 380-70-011, 727.011.005, 612.001.116, 277-70-004]
- Center for Creation, Content and Technology (CCCT)
- Hyperlocal Service Platform project
- Service Innovation ICT program
- WAHSP project
- BILAND project
- CLARIN-nl program
- Dutch national program COMMIT
- ESF Research Network Program ELIAS
Ask authors/readers for more resources
Expertise retrieval has attracted significant interest in the field of information retrieval. Expert finding has been studied extensively, with less attention going to the complementary task of expert profiling, that is, automatically identifying topics about which a person is knowledgeable. We describe a test collection for expert profiling in which expert users have self-selected their knowledge areas. Motivated by the sparseness of this set of knowledge areas, we report on an assessment experiment in which academic experts judge a profile that has been automatically generated by state-of-the-art expert-profiling algorithms; optionally, experts can indicate a level of expertise for relevant areas. Experts may also give feedback on the quality of the system-generated knowledge areas. We report on a content analysis of these comments and gain insights into what aspects of profiles matter to experts. We provide an error analysis of the system-generated profiles, identifying factors that help explain why certain experts may be harder to profile than others. We also analyze the impact on evaluating expert-profiling systems of using self-selected versus judged system-generated knowledge areas as ground truth; they rank systems somewhat differently but detect about the same amount of pairwise significant differences despite the fact that the judged system-generated assessments are more sparse.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available