Arbitrary-view human action recognition via novel-view action generation

Arbitrary-view human action recognition is still a big challenge due to the view changes. A possible solution is to enlarge the view range of action samples in the training set. Therefore, we propose a Two Branch Novel-View action Generation approach based on auxiliary conditional GAN, which generates a novel-view action sample for arbitrary-view human action recognition. The generated sample enlarge the view range of action samples for training. Furthermore, to narrow the representation of actions in different views, we propose a view-domain generalization model that improves the recognition performance of arbitrary-view human action recognition. Our approach is evaluated on three large-scale RGB+D skeleton datasets including UESTC varying-view RGB+D dataset, NTU RGB+D 60, and NTU RGB+D 120 datasets, with two types of view-invariant evaluations, i.e., the cross-view, and arbitrary-view recognition. The proposed approach achieves outstanding performance in human action recognition. (c) 2021 Elsevier Ltd. All rights reserved.

Automated summary

In this study, a Two Branch Novel-View action Generation approach based on auxiliary conditional GAN is proposed to generate novel-view action samples for enlarging the view range in the training set. Furthermore, a view-domain generalization model is introduced to improve the recognition performance of arbitrary-view human action recognition by narrowing the representation of actions in different views. The proposed approach achieves outstanding performance in human action recognition on three large-scale RGB+D skeleton datasets.