Riemannian representation learning for multi-source domain adaptation

Multi-Source Domain Adaptation (MSDA) aims at training a classification model that achieves small target error, by leveraging labeled data from multiple source domains and unlabeled data from a target domain. The source and target domains are described by related but different joint distributions, which lie on a Riemannian manifold named the statistical manifold. In this paper, we characterize the joint distribution difference by the Hellinger distance, which bears strong connection to the Riemannian metric defined on the statistical manifold. We show that the target error of a neural network classification model is upper bounded by the average source error of the model and the average Hellinger distance, i.e., the average of multiple Hellinger distances between the source and target joint distributions in the network represen-tation space. Motivated by the error bound, we introduce Riemannian Representation Learning (RRL): An approach that trains the network model by minimizing (i) the average empirical Hellinger distance with respect to the representation function, and (ii) the average empirical source error with respect to the network model. Specifically, we derive the average empirical Hellinger distance by constructing and solv-ing unconstrained convex optimization problems whose global optimal solutions are easy to find. With the network model trained, we expect it to achieve small error in the target domain. Our experimental results on several image datasets demonstrate that the proposed RRL approach is statistically better than the comparison methods. (c) 2022 Elsevier Ltd. All rights reserved.

Automated summary

Multi-Source Domain Adaptation (MSDA) aims to train a classification model that achieves small target error by utilizing labeled data from multiple source domains and unlabeled data from a target domain. This paper characterizes the joint distribution difference using Hellinger distance, and shows the upper bound of the target error of a neural network classification model. Motivated by the error bound, Riemannian Representation Learning (RRL) is introduced to train the network model by minimizing the average empirical Hellinger distance and empirical source error.