Fusing Higher-Order Features in Graph Neural Networks for Skeleton-Based Action Recognition

Skeleton sequences are lightweight and compact, thus are ideal candidates for action recognition on edge devices. Recent skeleton-based action recognition methods extract features from 3D joint coordinates as spatial-temporal cues, using these representations in a graph neural network for feature fusion to boost recognition performance. The use of first- and second-order features, \ie{} joint and bone representations, has led to high accuracy. Nonetheless, many models are still confused by actions that have similar motion trajectories. To address these issues, we propose fusing third-order features in the form of angular encoding into modern architectures to robustly capture the relationships between joints and body parts. This simple fusion with popular spatial-temporal graph neural networks achieves new state-of-the-art accuracy in two large benchmarks, including NTU60 and NTU120, while employing fewer parameters and reduced run time. Our source code is publicly available at: https://github.com/ZhenyueQin/Angular-Skeleton-Encoding.

Automated summary

Skeleton sequences are effective for action recognition on edge devices due to their lightweight and compact nature. By incorporating third-order features in the form of angular encoding into modern architectures, recognition performance can be improved while reducing parameters and run time. This fusion approach has achieved new state-of-the-art accuracy in large benchmarks such as NTU60 and NTU120.