4.7 Article

Temporal logic explanations for dynamic decision systems using anchors and Monte Carlo Tree Search

Journal

ARTIFICIAL INTELLIGENCE
Volume 318, Issue -, Pages -

Publisher

ELSEVIER
DOI: 10.1016/j.artint.2023.103897

Keywords

Interpretable machine learning; Black-box models; Anchors; Signal temporal logic; Monte Carlo Tree Search

Ask authors/readers for more resources

For complex automated perception and decision tasks, it is important to develop trust and understand the behavior of algorithms that may be too complex for human users. This article combines the anchors methodology with Monte Carlo Tree Search to provide explanations for black-box models' decisions. The methodology searches for descriptive explanations in the form of input signal properties, expressed in Signal Temporal Logic, to reproduce observed behavior.
For many automated perception and decision tasks, state-of-the-art performance may be obtained by algorithms that are too complex for their behavior to be completely understandable or predictable by human users, e.g., because they employ large machine learning models. To integrate these algorithms into safety-critical decision and control systems, it is particularly important to develop methods that can promote trust into their decisions and help explore their failure modes. In this article, we combine the anchors methodology with Monte Carlo Tree Search to provide local model-agnostic explanations for the behaviors of a given black-box model making decisions by processing time-varying input signals. Our approach searches for descriptive explanations for these decisions in the form of properties of the input signals, expressed in Signal Temporal Logic, which are highly likely to reproduce the observed behavior. To illustrate the methodology, we apply it in simulations to the analysis of a hybrid (continuous-discrete) control system and a collision avoidance system for unmanned aircraft (ACAS Xu) implemented by a neural network.(c) 2023 Elsevier B.V. All rights reserved.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.7
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

No Data Available
No Data Available