Journal
INFORMATION PROCESSING & MANAGEMENT
Volume 45, Issue 4, Pages 427-437Publisher
ELSEVIER SCI LTD
DOI: 10.1016/j.ipm.2009.03.002
Keywords
Performance evaluation; Machine Learning; Text classification
Funding
- Natural Sciences and Engineering Research Council of Canada
- Ontario Centres of Excellence
Ask authors/readers for more resources
This paper presents a systematic analysis of twenty four performance measures used in the complete spectrum of Machine Learning classification tasks, i.e., binary, multi-class, multi-labelled, and hierarchical. For each classification task, the study relates a set of changes in a confusion matrix to specific characteristics of data. Then the analysis concentrates on the type of changes to a confusion matrix that do not change a measure, therefore, preserve a classifier's evaluation (measure invariance). The result is the Measure invariance taxonomy with respect to all relevant label distribution changes in a classification problem. This formal analysis is supported by examples of applications where invariance properties of measures lead to a more reliable evaluation of classifiers. Text classification Supplements the discussion with several case studies. (C) 2009 Elsevier Ltd. All rights reserved.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available