期刊
INTERNATIONAL JOURNAL OF COMPUTER VISION
卷 130, 期 3, 页码 630-651出版社
SPRINGER
DOI: 10.1007/s11263-021-01546-9
关键词
Scene graphs; 3D scene understanding; Semantic segmentation
资金
- Projekt DEAL
A 3D scene is not only about geometry and object classes, but also about the semantic network of interconnected nodes. While scene graphs have been proven effective in image tasks, we propose a new neural network architecture to learn semantic graphs from 3D scenes. Our method goes beyond object-level perception and explores relations between object entities.
A 3D scene is more than the geometry and classes of the objects it comprises. An essential aspect beyond object-level perception is the scene context, described as a dense semantic network of interconnected nodes. Scene graphs have become a common representation to encode the semantic richness of images, where nodes in the graph are object entities connected by edges, so-called relationships. Such graphs have been shown to be useful in achieving state-of-the-art performance in image captioning, visual question answering and image generation or editing. While scene graph prediction methods so far focused on images, we propose instead a novel neural network architecture for 3D data, where the aim is to learn to regress semantic graphs from a given 3D scene. With this work, we go beyond object-level perception, by exploring relations between object entities. Our method learns instance embeddings alongside a scene segmentation and is able to predict semantics for object nodes and edges. We leverage 3DSSG, a large scale dataset based on 3RScan that features scene graphs of changing 3D scenes. Finally, we show the effectiveness of graphs as an intermediate representation on a retrieval task.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据