Does explainable machine learning uncover the black box in vision applications?

Machine learning (ML) in general and deep learning (DL) in particular has become an extremely popular tool in several vision applications (like object detection, super resolution, segmentation, object tracking etc.). Almost in parallel, the issue of explainability in ML (i.e. the ability to explain/elaborate the way a trained ML model arrived at its decision) in vision has also received fairly significant attention from various quarters. However, we argue that the current philosophy behind explainable ML suffers from certain limitations, and the resulting explanations may not meaningfully uncover black box ML models. To elaborate our assertion, we first raise a few fundamental questions which have not been adequately discussed in the corresponding literature. We also provide perspectives on how explainablity in ML can benefit by relying on more rigorous principles in the related areas.(c) 2021 Elsevier B.V. All rights reserved.

Automated summary

Machine learning and deep learning are widely used in vision applications, but the issue of explainability has limitations and may not effectively uncover black box models. To improve explainability, more rigorous principles in related areas should be relied upon.