Detecting Rotated Objects as Gaussian Distributions and its 3-D Generalization

Existing detection methods commonly use a parameterized bounding box (BBox) to model and detect (horizontal) objects and an additional rotation angle parameter is used for rotated objects. We argue that such a mechanism has fundamental limitations in building an effective regression loss for rotation detection, especially for high-precision detection with high IoU (e.g., 0.75). Instead, we propose to model the rotated objects as Gaussian distributions. A direct advantage is that our new regression loss regarding the distance between two Gaussians e.g., Kullback-Leibler Divergence (KLD), can well align the actual detection performance metric, which is not well addressed in existing methods. Moreover, the two bottlenecks i.e., boundary discontinuity and square-like problem also disappear. We also propose an efficient Gaussian metric-based label assignment strategy to further boost the performance. Interestingly, by analyzing the BBox parameters' gradients under our Gaussian-based KLD loss, we show that these parameters are dynamically updated with interpretable physical meaning, which help explain the effectiveness of our approach, especially for high-precision detection. We extend our approach from 2-D to 3-D with a tailored algorithm design to handle the heading estimation, and experimental results on twelve public datasets (2-D/3-D, aerial/text/face images) with various base detectors show its superiority.

Automated summary

Existing detection methods using parameterized bounding box and rotation angle have limitations for high-precision rotation detection. We propose modeling rotated objects as Gaussian distributions, with regression loss based on Kullback-Leibler Divergence to align detection performance. Our approach resolves boundary discontinuity and square-like problems, and uses an efficient Gaussian metric-based label assignment strategy for improved performance. Analysis of the BBox parameters' gradients shows their interpretable physical meaning, explaining the effectiveness of our approach. Extension to 3D with tailored algorithm design further enhances the performance, as demonstrated on various datasets.