Multi-scale skeleton adaptive weighted GCN for skeleton-based human action recognition in IoT

Skeleton-based human action recognition has become a hot topic due to its potential advantages. Graph convolution network (GCN) has obtained remarkable performances in the modeling of skeleton-based human action recognition in IoT. In order to capture robust spatial-temporal features from the human skeleton, a powerful feature extractor is essential. However, Most GCN-based methods use the fixed graph topology. Besides, only a single-scale feature is used, and the multi-scale information is ignored. In this paper, we propose a multi-scale skeleton adaptive weighted graph convolution network (MSAWGCN) for skeleton-based action recognition. Specifically, a multi-scale skeleton graph convolution network is adopted to extract more abundant spatial features of skeletons. Moreover, we develop a simple graph vertex fusion strategy, which can learn the latent graph topology adaptively by replacing the handcrafted adjacency matrix with a learnable matrix. According to different sampling strategies, weighted learning method is adopted to enrich features while aggregating. Experiments on three large datasets illustrate that the proposed method achieves comparable performances to state-of-the-art methods. Our proposed method attains an improvement of 0.9% and 0.7% respectively over the recent GCN-based method on the NTU RGB+D and Kinetics dataset. (c) 2021 Elsevier B.V. All rights reserved.

Automated summary

This paper proposes a multi-scale skeleton adaptive weighted graph convolution network (MSAWGCN) for skeleton-based action recognition, which extracts more abundant spatial features of skeletons through multi-scale skeleton graph convolution network and adopts a simple graph vertex fusion strategy to learn the graph topology adaptively.