Journal
APPLIED INTELLIGENCE
Volume 51, Issue 10, Pages 7151-7165Publisher
SPRINGER
DOI: 10.1007/s10489-021-02255-z
Keywords
Feature impact; Model interpretation; Visualization; Machine learning; Black box
Categories
Funding
- National Key R&D Program of China [2018YFC0830801]
Ask authors/readers for more resources
This study introduces a method to gain insight into black boxes of supervised learning models by visualizing the impacts of input features on prediction results. The method distinguishes typical impact patterns corresponding to different groups of observations, and reveals feature relationships embedded in prediction models.
Machine learning models have been widely used in various domains. However, the internal mechanisms of popular models, such as neural networks and support vector machines, are difficult for humans to understand; such models are often called black boxes. In this study, a general method is proposed to gain insight into the black boxes of supervised learning models by visualizing the impacts of input features on their prediction results. Compared with the existing methods, which may overlook the overall understanding of prediction models by analyzing the feature impacts for each individual observation or ignore the impact differences by providing a single impact pattern for all observations, the proposed method distinguishes some typical impact patterns that correspond to different groups of observations. The method maps the detected impact patterns into feature space using tree rules that help locate the impact patterns in the feature space. More importantly, the feature relationships embedded in the prediction models can be revealed through this tree rule-based feature relationship network. We apply the proposed method to various simulated and real data, and the results demonstrate how it can help us understand how features affect model prediction results and the relationships among features.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available