4.7 Article

A Multimodal Fusion Approach for Human Activity Recognition

期刊

出版社

WORLD SCIENTIFIC PUBL CO PTE LTD
DOI: 10.1142/S0129065723500028

关键词

Human activity recognition; multimodal fusion; deep convolutional neural networks

向作者/读者索取更多资源

In this paper, a multimodal approach for video-based human activity recognition (HAR) is proposed. It involves transforming 3D visual data into six different 2D image representations and extracting visual features using convolutional neural networks. The extracted features are then used for classification using a support vector machine. The proposed approach outperforms other state-of-the-art methods in most experiments.
The problem of human activity recognition (HAR) has been increasingly attracting the efforts of the research community, having several applications. It consists of recognizing human motion and/or behavior within a given image or a video sequence, using as input raw sensor measurements. In this paper, a multimodal approach addressing the task of video-based HAR is proposed. It is based on 3D visual data that are collected using an RGB+depth camera, resulting to both raw video and 3D skeletal sequences. These data are transformed into six different 2D image representations; four of them are in the spectral domain, another is a pseudo-colored image. The aforementioned representations are based on skeletal data. The last representation is a dynamic image which is actually an artificially created image that summarizes RGB data of the whole video sequence, in a visually comprehensible way. In order to classify a given activity video, first, all the aforementioned 2D images are extracted and then six trained convolutional neural networks are used so as to extract visual features. The latter are fused so as to form a single feature vector and are fed into a support vector machine for classification into human activities. For evaluation purposes, a challenging motion activity recognition dataset is used, while single-view, cross-view and cross-subject experiments are performed. Moreover, the proposed approach is compared to three other state-of-the-art methods, demonstrating superior performance in most experiments.

作者

我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。

评论

主要评分

4.7
评分不足

次要评分

新颖性
-
重要性
-
科学严谨性
-
评价这篇论文

推荐

暂无数据
暂无数据