Journal
PATTERN RECOGNITION
Volume 76, Issue -, Pages 80-94Publisher
ELSEVIER SCI LTD
DOI: 10.1016/j.patcog.2017.10.033
Keywords
Deep learning; Convolutional Neural Network; Recurrent neural network; Long Short-Term Memory; Human activity recognition; Hand gesture recognition; Real-time
Funding
- Spanish Government [MINECO/FEDER TIN2015-69542-C2-1, MINECO/ES TIN2014-57458-R]
- Banco de Santander
- Universidad Rey Juan Carlos Funding Program for Excellence Research Groups ref. Computer Vision and Image Processing (CVIP)
Ask authors/readers for more resources
In this work, we address human activity and hand gesture recognition problems using 3D data sequences obtained from full-body and hand skeletons, respectively. To this aim, we propose a deep learning-based approach for temporal 3D pose recognition problems based on a combination of a Convolutional Neural Network (CNN) and a Long Short-Term Memory (LSTM) recurrent network. We also present a two stage training strategy which firstly focuses on CNN training and, secondly, adjusts the full method (CNN+LSTM). Experimental testing demonstrated that our training method obtains better results than a single-stage training strategy. Additionally, we propose a data augmentation method that has also been validated experimentally. Finally, we perform an extensive experimental study on publicly available data benchmarks. The results obtained show how the proposed approach reaches state-of-the-art performance when compared to the methods identified in the literature. The best results were obtained for small datasets, where the proposed data augmentation strategy has greater impact. (C) 2017 Elsevier Ltd. All rights reserved.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available