Weighted boxes fusion: Ensembling boxes from different object detection models

Object detection is a crucial task in computer vision systems with a wide range of applications in autonomous driving, medical imaging, retail, security, face recognition, robotics, and others. Nowadays, neural networks based models are used to localize and classify instances of objects of particular classes. When real-time inference is not required, ensembles of models help to achieve better results. In this work, we present a novel method for fusing predictions from different object detection models: weighted boxes fusion. Our algorithm utilizes confidence scores of all proposed bounding boxes to construct averaged boxes. We tested the method on several datasets and evaluated it in the context of Open Images and COCO Object Detection challenges, achieving top results in these challenges. The 3D version of boxes fusion was successfully applied by the winning teams of Waymo Open Dataset and Lyft 3D Object Detection for Autonomous Vehicles challenges. The source code is publicly available at GitHub (Solovyev, 2019 [31]). We present a novel method for combining predictions in ensembles of different object detection models: weighted boxes fusion. This method significantly improves the quality of the fused predicted rectangles for an ensemble. We tested the method on several datasets and evaluated it in the context of the Open Images and COCO Object Detection challenges. It helped to achieve top results in these challenges. The source code is publicly available at GitHub. (c) 2021 Published by Elsevier B.V.

Automated summary

This study introduces a novel method, weighted boxes fusion, for combining predictions from different object detection models, significantly improving the quality of the ensemble predicted rectangles. The method achieved top results in various datasets and challenges, with the 3D version of boxes fusion being successfully applied in winning teams of specific competitions.