期刊
NEURAL NETWORKS
卷 167, 期 -, 页码 233-243出版社
PERGAMON-ELSEVIER SCIENCE LTD
DOI: 10.1016/j.neunet.2023.07.028
关键词
Domain invariance; Subpopulation shift; Joint distribution matching; Wasserstein distance; Neural networks; Supervised learning; Domain invariance; Subpopulation shift; Joint distribution matching; Wasserstein distance; Neural networks; Supervised learning
This paper addresses the domain shift problem in machine learning and proposes a method that can generate more invariant representations and more stable prediction performance across different domains, using mathematical relations with the Wasserstein distance. Empirical results on multiple image datasets show the effectiveness of the proposed approach.
Domain shifts in the training data are common in practical applications of machine learning; they occur for instance when the data is coming from different sources. Ideally, a ML model should work well independently of these shifts, for example, by learning a domain-invariant representation. However, common ML losses do not give strong guarantees on how consistently the ML model performs for different domains, in particular, whether the model performs well on a domain at the expense of its performance on another domain. In this paper, we build new theoretical foundations for this problem, by contributing a set of mathematical relations between classical losses for supervised ML and the Wasserstein distance in joint space (i.e. representation and output space). We show that classification or regression losses, when combined with a GAN-type discriminator between domains, form an upperbound to the true Wasserstein distance between domains. This implies a more invariant representation and also more stable prediction performance across domains. Theoretical results are corroborated empirically on several image datasets. Our proposed approach systematically produces the highest minimum classification accuracy across domains, and the most invariant representation. (c) 2023 The Author(s). Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据