Body part relation reasoning network for human activity understanding

The wide application of vision sensors has brought great opportunities and challenges to the intelligent understanding of image and video data. Research on effective human activ-ity understanding algorithms is of great significance in efficient processing and utilizing the data. At present, existing human activity understanding methods have the defects of low recognition accuracy, high complexity or poor robustness. In view of the above prob-lems, we construct the body part relation reasoning network (BPRRN). It aims to explore and utilize the common-sense knowledge in human bodies to reason human activities. Firstly, ten body part boxes are generated from each human body with the help of pose estimation and part classification functions. Secondly, with the states of each body part, the relationships between different body parts are explored and utilized for constructing the body part relation reasoning module (BPRRM). Finally, the body part relation reasoning features from each frame are adopted for temporal modeling with the temporal relation reasoning module (TRRM). In this way, the context association between video frames are constructed, which further promotes the reasoning ability of BPRRN. The experiments car-ried out on Stanford 40 and UCF101 demonstrate the effectiveness of the proposed network.(c) 2022 Elsevier Inc. All rights reserved.

Automated summary

The wide application of vision sensors has created both opportunities and challenges for intelligent understanding of image and video data. Efficient processing and utilization of this data necessitates research on effective algorithms for human activity understanding. Existing methods for human activity understanding suffer from issues such as low recognition accuracy, high complexity, and poor robustness. In response to these problems, the body part relation reasoning network (BPRRN) is constructed to leverage common-sense knowledge within human bodies for reasoning human activities. Experimental results on Stanford 40 and UCF101 datasets demonstrate the effectiveness of the proposed network.