Confidence Score: The Forgotten Dimension of Object Detection Performance Evaluation

When deploying a model for object detection, a confidence score threshold is chosen to filter out false positives and ensure that a predicted bounding box has a certain minimum score. To achieve state-of-the-art performance on benchmark datasets, most neural networks use a rather low threshold as a high number of false positives is not penalized by standard evaluation metrics. However, in scenarios of Artificial Intelligence (AI) applications that require high confidence scores (e.g., due to legal requirements or consequences of incorrect detections are severe) or a certain level of model robustness is required, it is unclear which base model to use since they were mainly optimized for benchmark scores. In this paper, we propose a method to find the optimum performance point of a model as a basis for fairer comparison and deeper insights into the trade-offs caused by selecting a confidence score threshold.

Automated summary

When deploying models for object detection, choosing the confidence score threshold is crucial to balance between filtering out false positives and ensuring a minimum score for predicted bounding boxes. While neural networks often use low thresholds for state-of-the-art performance, it remains unclear how to select base models optimized for benchmark scores when high confidence scores or robustness are needed for AI applications.