Spiking neural network based on joint entropy of optical flow features for human action recognition

In the recent past, human action recognition is inviting increased attention in the automated video surveillance systems. An efficient human action classification technique in an unconstrained environment is proposed in this paper. A novel descriptor relying on joint entropy of difference in magnitude and orientation of the optical flow feature is developed in order to model human actions. Initially, flow feature is computed using Pyramid-Warping-Cost volume Network (PWCNet), considering every two consecutive frames. Then, the feature descriptor is formed based on the joint entropy of difference in flow magnitude and orientation collected from the regular grid of each frame in the action sequence. Finally, in order to incorporate long-term temporal dependency, a spiking neural network is embedded to aggregate the information across the frames. Different optimization techniques and different types of hidden nodes are utilized in the spiking neural network to analyze the performance of the proposed work. Extensive experiments on the benchmark datasets for human action recognition show the efficacy of the proposed method.

Automated summary

In this paper, an efficient technique for human action recognition in automated video surveillance systems is proposed. The technique utilizes optical flow features and joint entropy to model human actions, and incorporates a spiking neural network to aggregate information across frames. Experimental results demonstrate the effectiveness of the proposed method.