期刊
IEEE ACCESS
卷 10, 期 -, 页码 111725-111731出版社
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/ACCESS.2022.3214812
关键词
Convolutional neural networks; Training; Three-dimensional displays; Correlation; Pose estimation; Feature extraction; Data models; Skeletons; Action recognition; attention mechanism; feature fusion; graph convolutional networks; human skeleton; pose information
资金
- Major Project of the Korea Institute of Civil Engineering and Building Technology (KICT) [20220238-001]
PG-GCN is a multi-modal framework that explores robust features from both pose and skeleton data simultaneously, with early-stage feature fusion using a dynamic attention module, achieving state-of-the-art performance in action recognition tasks.
Graph convolutional networks (GCN), which can model the human body skeletons as spatial and temporal graphs, have shown remarkable potential in skeleton-based action recognition. However, in the existing GCN-based methods, graph-structured representation of the human skeleton makes it difficult to be fused with other modalities, especially in the early stages. This may limit their scalability and performance in action recognition tasks. In addition, the pose information, which naturally contains informative and discriminative clues for action recognition, is rarely explored together with skeleton data in existing methods. In this work, we proposed pose-guided GCN (PG-GCN), a multi-modal framework for high-performance human action recognition. In particular, a multi-stream network is constructed to simultaneously explore the robust features from both the pose and skeleton data, while a dynamic attention module is designed for early-stage feature fusion. The core idea of this module is to utilize a trainable graph to aggregate features from the skeleton stream with that of the pose stream, which leads to a network with more robust feature representation ability. Extensive experiments show that the proposed PG-GCN can achieve state-of-the-art performance on the NTU RGB+D 60 and NTU RGB+D 120 datasets.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据