Multiple-instance domain adaptation for cost-effective sensor-based human activity recognition

Machine learning-based human activity recognition (HAR) is important as the means of human- computer interaction to empower the existing systems in many areas, such as healthcare, entertainment, logistics, and manufacturing. To build such a recognition tool, it is clear that sufficient labeled samples are required. Oftentimes, it is more difficult to obtain labeled samples rather than to obtain unlabeled ones due to the prohibitive conditions (e.g., financial cost, time, hazardous environment, and human labor), so we end up with incomplete or weakly labeled data. The multiple-instance learning technique (MIL) alleviates such issue by allowing us to leverage weakly labeled data by performing the classification of a bag of instances rather than a single instance. However, since multiple-instance learning is intrinsically the generalization of supervised learning, it may face the same problem as the usual supervised learning approaches: performance degradation on the data with different distribution. In fact, such distribution difference is common in sensor-based HAR which makes it difficult for a classifier model to perform predictions. For example, the difficulties happen when the current data distribution is shifted due to sensor deterioration, or when the model that is generated from a certain domain is applied to a different domain (e.g., different person with different device placement, posture, and gait). In this work, we propose a multiple-instance domain adaptation approach that handles weakly annotated data for model training, while providing adaptation mechanism to deal with data distribution difference. We incorporate high-level adaptation and bag-level adaptation to find a robust sensor data representation which minimizes distribution difference. The proposed approach is tested on standard sensor-based HAR datasets, conditioned on weak annotation and cross-domain settings. Our experimental result shows promising recognition performance improvements compared to the classical MIL and domain adaptation approaches. (C) 2022 Elsevier B.V. All rights reserved.

Automated summary

Machine learning-based human activity recognition is important for human-computer interaction in various fields. Multiple-instance learning techniques can leverage weakly labeled data for classification. However, the performance may degrade on data with different distribution, which is common in sensor-based activity recognition.