Tripool: Graph triplet pooling for 3D skeleton-based action recognition

Graph Convolutional Network (GCN) has already been successfully applied to skeleton-based action recog-nition. However, current GCNs in this task are lack of pooling operations such that the architectures are inherently flat, which not only increases the computational complexity but also requires larger memory space to keep the entire graph embedding. More seriously, a flat architecture forces the high-level seman-tic feature representations to have the same physical structure of the low-level input skeletons, which we argue is unreasonable and harmful for the final performance. To address these issues, we propose Tripool, a novel graph pooling method for 3D action recognition from skeleton data. Tripool provides to optimize a triplet pooling loss, in which both graph topology and global graph context are taken into considera-tion, to learn a hierarchical graph representation. The training process of graph pooling is efficient since it optimizes the graph topology by minimizing an upper bound of the pooling loss. Besides, Tripool also automatically generates an embedding matrix since the graph is changed after pooling. On one hand, Tripool reduces the computational cost by removing the redundant nodes. On the other hand it over-comes the limitation of the topology constrain for the high-level semantic representations, thus improves the final performance. Tripool can be combined with various graph neural networks in an end-to-end fashion. Comprehensive experiments on two current largest scale 3D datasets are conducted to evalu-ate our method. With our Tripool, we consistently get the best results in terms of various performance measures. (C) 2021 The Author(s). Published by Elsevier Ltd.

Automated summary

Graph Convolutional Network (GCN) is successfully applied to skeleton-based action recognition, however, lacking pooling operations leads to flat architectures, Tripool offers a solution for this issue.