DB-LSTM: Densely-connected Bi-directional LSTM for human action recognition

Although deep learning has achieved promising progress recently, action recognition remains a challeng-ing task, due to cluttered backgrounds, diverse scenes, occlusions, viewpoint variations and camera motions. In this paper, we propose a novel deep learning model to capture the spatial and temporal pat-terns of human actions from videos. Sample representation learner is proposed to extract the video-level temporal feature, which combines the sparse temporal sampling and long-range temporal learning to form an efficient and effective training strategy. To boost the effectiveness and robustness of modeling long-range action recognition, a Densely-connected Bi-directional LSTM (DB-LSTM) network is novelly proposed to model the visual and temporal associations in both forward and backward directions. They are stacked and integrated with the dense skip-connections to improve the capability of temporal pattern modeling. Two modalities from appearance and motion are integrated with a fusion module to further improve the performance. Experiments conducted on two benchmark datasets, UCF101 and HMDB51, demonstrate that the proposed DB-LSTM model achieves promising performance, which out-performs the state-of-the-art approaches for action recognition. (c) 2020 Elsevier B.V. All rights reserved.

Automated summary

A novel deep learning model is proposed in this paper to capture spatial and temporal patterns of human actions from videos, utilizing methods such as sample representation learner, Densely-connected Bi-directional LSTM network, and fusion of appearance and motion modalities. These techniques improve the effectiveness and robustness of long-range action recognition, leading to promising performance surpassing existing approaches in benchmark datasets UCF101 and HMDB51.