Improved Surprise Adequacy Tools for Corner Case Data Description and Detection

Facing the increasing quantity of AI models applications, especially in life- and property-related fields, it is crucial for designers to construct safety- and security-critical systems. As a major factor affecting the safety of AI models, corner case data and its related description/detection techniques are important in the AI design phase and quality assurance. In this paper, inspired by surprise adequacy (SA), a tool having advantages on capture data behaviors, we developed three modified versions of distance-based-SA (DSA) for detecting corner cases in classification problems. Through the experiment analysis on MNIST, CIFAR, and industrial example data, the feasibility and usefulness of the proposed tools on corner case data detection are verified. Moreover, Qualitative and quantitative experiments validated that the developed DSA tools can achieve improved performance in describing corner cases' behaviors.

Automated summary

In the face of increasing applications of AI models, it is crucial for designers to construct safety-critical systems, especially in life- and property-related fields. Corner case data, as a major factor affecting the safety of AI models, and its related detection techniques play an important role in the AI design phase and quality assurance. This paper introduces three modified versions of distance-based-SA (DSA) for detecting corner cases in classification problems, showing feasibility and usefulness through experiment analysis on various datasets. The developed DSA tools demonstrate improved performance in describing corner cases' behaviors through qualitative and quantitative experiments.