Generating comprehensive scene graphs with integrated multiple attribute detection

Scene graphs present the semantic highlights of the underlying image in directed graph form. Their automated generation is often restricted to detecting multiple relations for objects. The diverse attributes of the objects are neglected in the process. We propose a Multiple Attribute Detector that can capture structured attribute information of an object, i.e. attribute type, value in the form of a triplet. The module is capable of generating multiple such triplets for every detected object in the scene. It can be integrated with existing scene graph generation frameworks without altering relation detection mechanism to yield comprehensive scene graphs. We have also created new datasets for this purpose that include attribute type-value data per object.

Automated summary

Scene graphs present the semantic highlights of the underlying image in directed graph form. Their automated generation often neglects the diverse attributes of the objects. We propose a Multiple Attribute Detector that can capture structured attribute information of an object, including attribute type and value. This module can generate multiple triplets for each detected object in the scene and be integrated with existing scene graph generation frameworks.