Journal
PATTERN RECOGNITION
Volume 139, Issue -, Pages -Publisher
ELSEVIER SCI LTD
DOI: 10.1016/j.patcog.2023.109428
Keywords
Temporal attention; Spatial attention; Spatio-temporal attention; Activity recognition; Skeleton data; Deep reinforcement learning
Ask authors/readers for more resources
The use of skeleton data for activity recognition has become widespread due to its advantages over RGB data. In this paper, a novel framework called STH-DRL is proposed for activity recognition, which includes a temporal agent and a spatial agent. The agents are trained using deep reinforcement learning to find key frames and key joints, respectively, by formulating the search problems as Markov decision processes. Experimental results on three benchmark datasets demonstrate the effectiveness of the proposed method.
The use of skeleton data for activity recognition has become prevalent due to its advantages over RGB data. A skeleton video includes frames showing two-or three-dimensional coordinates of human body joints. For recognizing an activity, not all the video frames are informative, and only a few key frames can well represent an activity. Moreover, not all joints participate in every activity; i.e., the key joints may vary across frames and activities. In this paper, we propose a novel framework for finding temporal and spatial attentions in a cooperative manner for activity recognition. The proposed method, which is called STH-DRL, consists of a temporal agent and a spatial agent. The temporal agent is responsible for finding the key frames, i.e., temporal hard attention finding, and the spatial agent attempts to find the key joints, i.e., spatial hard attention finding. We formulate the search problems as Markov decision processes and train both agents through interacting with each other using deep reinforcement learning. Experimental results on three widely used activity recognition benchmark datasets demonstrate the effectiveness of our proposed method. Crown Copyright (c) 2023 Published by Elsevier Ltd. All rights reserved.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available