RGB-D-based categorical object pose and shape estimation: Methods, datasets, and evaluation

Recently, various methods for 6D pose and shape estimation of objects at a per-category level have been proposed. This work provides an overview of the field in terms of methods, datasets, and evaluation protocols. First, an overview of existing works and their commonalities and differences is provided. Second, we take a critical look at the predominant evaluation protocol, including metrics and datasets. Based on the findings, we propose a new set of metrics, contribute new annotations for the Redwood dataset, and evaluate state-of-the-art methods in a fair comparison. The results indicate that existing methods do not generalize well to unconstrained orientations and are actually heavily biased towards objects being upright. We provide an easy-to-use evaluation toolbox with welldefined metrics, methods, and dataset interfaces, which allows evaluation and comparison with various state-of-the-art approaches (https://github.com/roym899/pose_and_shape_evaluation).(c) 2023 The Author(s). Published by Elsevier B.V. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).

Automated summary

This paper provides an overview of methods, datasets, and evaluation protocols for 6D pose and shape estimation of objects at a per-category level. It presents the commonalities and differences of existing works, critiques the prevailing evaluation protocol, and proposes new metrics and annotations. The study finds that existing methods struggle with unconstrained orientations and have a bias towards upright objects. The authors also provide an easy-to-use evaluation toolbox.