Journal
INFORMATION SCIENCES
Volume 181, Issue 13, Pages 2685-2699Publisher
ELSEVIER SCIENCE INC
DOI: 10.1016/j.ins.2010.05.029
Keywords
Data matching; Similarity function; Quality estimation
Categories
Funding
- Brazilian National Institute of Science and Technology for the Web (CNPq) [573871/2008-6]
- CNPq [550891/2007-2]
Ask authors/readers for more resources
Several advanced data management applications, such as data integration, data deduplication, and similarity querying rely on the application of similarity functions. A similarity function requires the definition of a threshold value in order to decide whether two different data instances match, i.e., if they represent the same real world object. In this context, threshold definition is a central problem. This paper proposes a method for estimating the quality of a similarity function. Quality is measured in terms of recall and precision calculated at several different thresholds. Based on the results of the proposed estimation process and the requirements of a specific application, a user is able to choose a suitable threshold value. The estimation process is based on a clustering phase performed over a data collection (or a sample thereof) and requires no human intervention since the choice of similarity threshold is based on the silhouette coefficient, which is an internal quality measure for clusters. An extensive set of experiments on artificial and real datasets demonstrates the effectiveness of the proposed approach. The results of the experiments show that in most cases the estimation error was below 10% in terms of precision and recall. (C) 2010 Elsevier Inc. All rights reserved.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available