期刊
MACHINE LEARNING
卷 111, 期 5, 页码 1903-1923出版社
SPRINGER
DOI: 10.1007/s10994-022-06167-y
关键词
Explainable AI; Saliency methods; Ground truth; Benchmark; Linear classification; Suppressor variables
资金
- European Research Council (ERC) under the European Union [758985]
- German Ministry for Education and Research [01IS18025A, 01IS18037A]
- German Research Foundation (DFG) as Math+: Berlin Mathematics Research Center [EXC 2046/1, 390685689]
- Institute of Information & Communications Technology Planning & Evaluation (IITP) - Korea Government [2019-0-00079]
- Artificial Intelligence Graduate School Program, Korea University
- European Research Council (ERC) [758985] Funding Source: European Research Council (ERC)
Machine learning is increasingly used in high-stakes decision-making, but complex ML models are often considered black boxes. This has led to the field of explainable AI (XAI), which aims to shed light on the inner workings of these models. Saliency methods are commonly used to rank input features, but validating their results is challenging. This study proposes a definition for feature importance and evaluates various explanation methods using a benchmark dataset.
Machine learning (ML) is increasingly often used to inform high-stakes decisions. As complex ML models (e.g., deep neural networks) are often considered black boxes, a wealth of procedures has been developed to shed light on their inner workings and the ways in which their predictions come about, defining the field of 'explainable AI' (XAI). Saliency methods rank input features according to some measure of 'importance'. Such methods are difficult to validate since a formal definition of feature importance is, thus far, lacking. It has been demonstrated that some saliency methods can highlight features that have no statistical association with the prediction target (suppressor variables). To avoid misinterpretations due to such behavior, we propose the actual presence of such an association as a necessary condition and objective preliminary definition for feature importance. We carefully crafted a ground-truth dataset in which all statistical dependencies are well-defined and linear, serving as a benchmark to study the problem of suppressor variables. We evaluate common explanation methods including LRP, DTD, PatternNet, PatternAttribution, LIME, Anchors, SHAP, and permutation-based methods with respect to our objective definition. We show that most of these methods are unable to distinguish important features from suppressors in this setting.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据