Human-object interaction detection with missing objects

Human-object interaction (HOI) detection is an important vision task that requires the detection of individual object instances and reasoning of their relations. Despite encouraging advancement in recent years, past methods are still limited to relatively simple images where the human and object instances can be detected without difficulties. HOI in the wild should work even when the objects that a person is interacting with are not visible or hard to detect in the image. In this paper, we formulate HOI with missing objects (HOI-MO) as a research problem, and show that it is indeed critical as many such instances can be found even in the commonly-used public HOI detection datasets. We label these to compose new test sets for the proposed method. To our knowledge, we introduce the first method for such challenging HOI detection that incorporates global scene information. The effectiveness and superiority of the proposed method are demonstrated through extensive experiments and comparisons. (c) 2021 Elsevier B.V. All rights reserved.

Automated summary

Human-object interaction (HOI) detection is a crucial vision task that involves detecting individual object instances and reasoning their relations. The challenge lies in detecting HOI instances with missing objects, which are prevalent in commonly-used public datasets. This paper introduces a novel method that incorporates global scene information for effective and superior HOI detection performance.