期刊
PATTERN RECOGNITION LETTERS
卷 116, 期 -, 页码 150-156出版社
ELSEVIER SCIENCE BV
DOI: 10.1016/j.patrec.2018.10.005
关键词
Multiple annotators; Kernel methods; Classification
资金
- COLCIENCIAS [1110-744-55958]
In a typical supervised learning scenario, it is supposed that there is an oracle who gives the correct label (also known as gold standard or ground truth) for each instance available in the training set. Nevertheless, for many real-world problems, instead of the gold standard, we have access to some annotations (possibly noisy) provided by multiple annotators with different unknown levels of expertise. Then, it is not appropriate to use trivial methods, i.e., majority voting, to estimate the actual label from the annotations due to this way assumes homogeneity in the performance of the labelers. Here, we introduce a new kernel alignment-based annotator relevance analysis-(KAAR) approach to code each annotator expertise as an averaged matching between the input features and the expert labels. So, a new sample label is predicted as a convex combination of classifiers adopting the achieved KAAR-based coding. Experimental results show that our methodology can estimate the performance of annotators even if the gold standard is not available, defeating state-of-the-art techniques. (C) 2018 Elsevier B.V. All rights reserved.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据