4.7 Article

SGM-Net: Skeleton-guided multimodal network for action recognition

期刊

PATTERN RECOGNITION
卷 104, 期 -, 页码 -

出版社

ELSEVIER SCI LTD
DOI: 10.1016/j.patcog.2020.107356

关键词

Action recognition; multi-modality; skeleton-guided

资金

  1. National Natural Science Foundation of China [61836008, 61632019]

向作者/读者索取更多资源

Single-modality human action recognition on RGB or skeleton has been extensively studied. Each of these two modalities has its own advantages as well as limitations, because they depict action from different perspectives. The feature of different modalities can complement each other for describing actions. Therefore, it is meaningful to fuse these two modalities using their complementarity for action recognition. However, existing multimodal methods fail to fully exploit the complementarity of RGB and skeleton modalities. In this paper, we propose a Skeleton-Guided Multimodal Network (SGM-Net) for human action recognition. The proposed method takes full use of the complementarity of these two modalities at semantic feature level. From the technical perspective, we introduce a guided block, the key component of SGM-Net. It enables skeleton feature to guide on RGB feature, so that the important RGB information strongly related to the action is enhanced. Moreover, in the guided block, two schemes of correlation operation are explored. We perform a series of ablation experiments to verify the effectiveness of the guided block. The experimental results show that our approach achieves state-of-the-art performance over the existing methods on NTU and Sub-JHMDB datasets. (C) 2020 Elsevier Ltd. All rights reserved.

作者

我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。

评论

主要评分

4.7
评分不足

次要评分

新颖性
-
重要性
-
科学严谨性
-
评价这篇论文

推荐

暂无数据
暂无数据