Video anomaly detection based on 3D convolutional auto-encoder

Video anomaly detection plays a critical role in public safety and security. However, it is hard to perform supervised due to its characteristics such as definition ambiguity, scene dependency, and sample scarcity. This paper proposes an unsupervised video anomaly detection model, called Spatio-Temporal 3D Convolutional Auto-Encoder model (ST-3DCAE) based on the input of the fused features of both motion and appearance. First, to utilize both motion and appearance information in the scene, the optical flow feature map of the scene is extracted with PWCNet and fused with the original video frame as the model input. Then, the 3DConv module and the Convolution Long Short-Term Memory(ConvLSTM) module are then used for extracting the spatio-temporal features, and the 3DSEblock module is used to screen important features. Finally, the reconstruction error between the input and output of the auto-encoder is used to determine whether the video frames are related to abnormal behavior. The proposed model has been validated on publicly available datasets such as UCSD Pedestrian and Avenue datasets. Experimental results analysis, both qualitative and quantitative, demonstrate the effectiveness of the proposed method.

Automated summary

This paper proposes an unsupervised video anomaly detection model called ST-3DCAE, which combines motion and appearance features. The model uses optical flow feature maps fused with original video frames for input, and extracts spatio-temporal features using 3DConv and ConvLSTM modules. The 3DSEblock module is used to screen important features, and the reconstruction error of the auto-encoder determines the relevance of video frames to abnormal behavior. Experimental results demonstrate the effectiveness of the proposed method.