Learning Disentangled Representation for Mixed-Reality Human Activity Recognition With a Single IMU Sensor

Together with the rapid development of the sensors technology in recent years, sensor-based human activity recognition (HAR) has shown promising performance using well-known supervised deep learning methods. However, it remains challenging in a realistic scenario, i.e., limited number of labeled samples and sensors. This article proposes a novel deep learning method to achieve accurate and robust HAR with only a single inertial measurement unit (IMU) sensor. Our contributions are twofold. First, based on the skinned multiperson linear (SMPL) model, we build a large synthetic HAR dataset containing multimodal measurements: acceleration and angular velocity, which were generated according to the forward kinematics. Second, We propose a multiple-level domain adaptive learning model with information-theoretically stimulated constraints to simultaneously align the distributions of low- and high-level representations of virtual and real HAR data. The proposed mutual information constraints encourage the neural network to learn a disentangled representation for the multimodal sensing data. Comprehensive experimental results on three publicly available datasets demonstrate that the proposed method compares favorably with competing ones and has robust performance with variable labeled samples.

Automated summary

This article introduces a novel deep learning method for accurate and robust human activity recognition with only using a single IMU sensor. By constructing a large synthetic dataset with multimodal measurements and proposing a multiple-level domain adaptive learning model, the neural network is encouraged to learn a disentangled representation for multimodal sensing data, achieving favorable performance in comparison with competing methods.