Abnormal sitting posture recognition based on multi-scale spatiotemporal features of skeleton graph

To fuse the spatial dimension, temporal dimension, and whole-body skeleton features of human abnormal sitting posture to improve the recognition accuracy, we propose an abnormal sitting posture recognition method based on the multi-scale spatiotemporal features of skeleton graph (ASPR) from the perspective of changes of human posture. Firstly, we build a human abnormal sitting posture dataset (HASP) with multidimensional features. Then, based on the structure of high-to-low resolution subnetworks, we use the spatiotemporal graph convolution module as the feature extraction unit A multiple scale spatiotemporal feature extraction model based on graph convolutional network (M2SGCN) is proposed. It is used to capture the spatiotemporal features. Next, a feature extraction model of local skeletal angles based on recurrent neural network is proposed to capture the change rule of skeletal angles of human sitting posture. Finally, we investigate the optimal parameters of ASPR by checking its performance with different human skeleton combination schemes and different features fusion coefficient. Experiment results show that ASPR achieves excellent performance compared with four state-of-the-art models and their combined models. ASPR shows an average recognition accuracy, a recall rate, an F1 score, and average time of 95.24%, 95.61%, 95.14%, and 7.094 ms, respectively.

Automated summary

In this study, we propose an abnormal sitting posture recognition method based on the multi-scale spatiotemporal features of skeleton graph (ASPR) to fuse the spatial dimension, temporal dimension, and whole-body skeleton features. We first build a human abnormal sitting posture dataset (HASP) with multidimensional features. Then, we propose a multiple scale spatiotemporal feature extraction model based on graph convolutional network (M2SGCN) to capture the spatiotemporal features. Additionally, a feature extraction model of local skeletal angles based on recurrent neural network is used to capture the change rule of skeletal angles of human sitting posture. Experimental results demonstrate the superior performance of ASPR compared to state-of-the-art models.