期刊
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
卷 44, 期 7, 页码 3508-3522出版社
IEEE COMPUTER SOC
DOI: 10.1109/TPAMI.2021.3055780
关键词
Semantics; Biological system modeling; Cognition; Task analysis; Legged locomotion; Computational modeling; Message passing; Human parsing; hierarchical model; relation reasoning; graph neural network
资金
- CCF-Baidu Open Fund
- Zhejiang Lab's Open Fund [2019KD0AB04]
This study examines three types of inference processes in human parsing, proposes a compositional neural information fusion framework, and develops a part-relation-aware human parser. The results demonstrate superior performance of the new methods in human parsing tasks.
Modeling the human structure is central for human parsing that extracts pixel-wise semantic information from images. We start with analyzing three types of inference processes over the hierarchical structure of human bodies: direct inference (directly predicting human semantic parts using image information), bottom-up inference (assembling knowledge from constituent parts), and top-down inference (leveraging context from parent nodes). We then formulate the problem as a compositional neural information fusion (CNIF) framework, which assembles the information from the three inference processes in a conditional manner, i.e., considering the confidence of the sources. Based on CNIF, we further present a part-relation-aware human parser (PRHP), which precisely describes three kinds of human part relations, i.e., decomposition, composition, and dependency, by three distinct relation networks. Expressive relation information can be captured by imposing the parameters in the relation networks to satisfy specific geometric characteristics of different relations. By assimilating generic message-passing networks with their edge-typed, convolutional counterparts, PRHP performs iterative reasoning over the human body hierarchy. With these efforts, PRHP provides a more general and powerful form of CNIF, and lays the foundation for more sophisticated and flexible human relation patterns of reasoning. Experiments on five datasets demonstrate that our two human parsers outperform the state-of-the-arts in all cases.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据