A spatiotemporal multi-feature extraction framework with space and channel based squeeze-and-excitation blocks for human activity recognition

Human activity recognition (HAR) is an active field in ubiquitous computing and body area network (BAN), which has been widely applied in medical care, sport and smart home. In recent years, a lot of methods based on deep learning show great performance on HAR. In consideration of the temporal and spatial dependencies of time series, the extracted features of traditional methods are not comprehensive. In this paper, we propose a new activity recognition framework based on spatiotemporal multi-feature extraction with space and channel based squeeze-and-excitation blocks (SCbSE-SMFE). The framework includes a temporal feature extraction layer composed of gated recurrent unit (GRU) blocks, a spatial feature extraction layer composed of convolutional neural networks (CNN) blocks with SCbSE blocks, a statistical feature extraction layer and an output layer. Meanwhile, regarding the actual needs for recognizing aggressive activities, we simulate the prison environment and collect an aggressive activity dataset (AAD). What's more, aiming at the characteristics of aggressive activities, a threshold-based aggressive activity detection method is proposed to reduce the computational complexity. The proposed framework is evaluated on the public dataset WISDM and the collected dataset AAD, and the results prove that the proposed SCbSE-SMFE framework can effectively improve the accuracy and distinguish similar activities better. The proposed aggressive activity detection method based on threshold can simplify the model and improve the recognition speed while ensuring the recognition accuracy.

Automated summary

This paper proposes a new activity recognition framework based on spatiotemporal multi-feature extraction with SCbSE blocks. By simulating the prison environment and collecting an aggressive activity dataset, a threshold-based aggressive activity detection method is developed to simplify the model and enhance recognition speed.