Jointly Identifying and Fixing Inconsistent Readings from Information Extraction Systems

Information extraction systems analyze text to produce entities and beliefs, but their output often has errors. In this paper we analyze the reading consistency of the extracted facts with respect to the text from which they were derived and show how to detect and correct errors. We consider both the scenario when the provenance text is automatically found by an IE system and when it is curated by humans. We contrast consistency with credibility; define and explore consistency and repair tasks; and demonstrate a simple, yet effective and generalizable, model. We analyze these tasks and evaluate this approach on three datasets. Against a strong baseline model, we consistently improve both consistency and repair across three datasets using a simple MLP model with attention and lexical features.

Automated summary

This paper investigates the problem of errors in information extraction systems' outputs and explores methods to detect and correct these errors. The authors contrast consistency with credibility, define and explore consistency and repair tasks, and present a simple yet effective model. Evaluation on three datasets shows consistent improvement in both consistency and repair using a simple MLP model with attention and lexical features.