Efficient probability intervals for classification using inductive venn predictors

Learning enabled components are frequently used by autonomous systems and it is common for deep neural networks to be integrated in such systems for their ability to learn complex, non-linear data pat-terns and make accurate predictions in dynamic environments. However, their large number of parame-ters and their use as black boxes introduce risks as the confidence in each prediction is unknown and out-put values like softmax scores are not usually well-calibrated. Different frameworks have been proposed to compute accurate confidence measures along with the predictions but at the same time introduce a number of limitations like execution time overhead or inability to be used with high-dimensional data. In this paper, we use the Inductive Venn Predictors framework for computing probability intervals regarding the correctness of each prediction in real-time. We propose taxonomies based on distance metric learning to compute informative probability intervals in applications involving high-dimensional inputs. By assign-ing pseudo-labels to unlabeled input data during system deployment we further improve the efficiency of the computed probability intervals. Empirical evaluation on image classification and botnet attacks de-tection in Internet-of-Things (IoT) applications demonstrates improved accuracy and calibration. The pro-posed method is computationally efficient, and therefore, can be used in real-time. The code is available at https://github.com/dboursinos/Efficient-Probability-Intervals-Classification-Inductive-Venn-Predictors .& COPY; 2023 Elsevier Ltd. All rights reserved.

Automated summary

This paper presents a method for computing probability intervals in real-time, which assigns pseudo-labels to unlabeled input data to improve efficiency. Empirical evaluation shows that the proposed method improves accuracy and calibration in image classification and botnet attack detection in IoT applications.