期刊
ACM TRANSACTIONS ON KNOWLEDGE DISCOVERY FROM DATA
卷 16, 期 3, 页码 -出版社
ASSOC COMPUTING MACHINERY
DOI: 10.1145/3477539
关键词
Attention mechanism; model interpretation; document classification; sentiment classification; concept-based explanation
资金
- US National Science Foundation [IIS-1707498, IIS-1838730]
This study introduces an explanation method that captures causal relationships between keywords and model predictions by learning the importance of keywords for predicted labels across a training corpus based on attention weights. It can automatically learn higher-level concepts and their importance to model prediction tasks.
Using attention weights to identify information that is important for models' decision making is a popular approach to interpret attention-based neural networks. This is commonly realized in practice through the generation of a heat-map for every single document based on attention weights. However, this interpretation method is fragile and it is easy to find contradictory examples. In this article, we propose a corpus-level explanation approach, which aims at capturing causal relationships between keywords and model predictions via learning the importance of keywords for predicted labels across a training corpus based on attention weights. Based on this idea, we further propose a concept-based explanation method that can automatically learn higher level concepts and their importance to model prediction tasks. Our concept-based explanation method is built upon a novel Abstraction-Aggregation Network (AAN), which can automatically cluster important keywords during an end-to-end training process. We apply these methods to the document classification task and show that they are powerful in extracting semantically meaningful keywords and concepts. Our consistency analysis results based on an attention-based Naive Bayes classifier (NBC) also demonstrate that these keywords and concepts are important for model predictions.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据