Transferable regularization and normalization: Towards transferable feature learning for unsupervised domain adaptation

Unsupervised domain adaptation aims at alleviating distribution discrepancy when trans-ferring knowledge from a large-scale labeled source domain to a new unlabeled target domain. A prevailing method is adversarial feature adaptation, which generates transfer-able features via aligning feature distributions across domains to bound domain discrep-ancy. However, such approach renders suboptimal performances and vulnerable to negative transfer when target domain has less number of classes compared to source domain as in partial domain adaptation. To address this issue, previous methods adopt a transferable feature norm strategy to adapting feature norms of two domains achieving significant transfer gains. Nonetheless, these works do not consider the intrinsic limitation of the architecture design of deep neural networks which greatly influences the loss func-tion of transferability. In this paper, we propose Transferable Regularization and Normalization (TRN), which simultaneously avoids negative transfer via adapting feature norms of both domains and facilitates positive transfer via replacing the existing normal-ization techniques in mainstream deep backbones. As a general approach, TRN can be sim-ply embedded into deep transfer learning approaches. After a thorough evaluation of proposed method utilizing several benchmark datasets (Office-31, ImageCLEF-DA, Office -Home and VisDA-2017). TRN yielded the state-of-the-art results and outperformed other approaches (IFAN, PADA) by a large margin (0.5%, 7.74%) for various domain adaptation tasks. (c) 2022 Elsevier Inc. All rights reserved.

Automated summary

This paper proposes a method called Transferable Regularization and Normalization (TRN) for unsupervised domain adaptation. TRN adjusts feature norms and improves normalization techniques to avoid negative transfer and facilitate positive transfer. Evaluation results show that TRN achieves state-of-the-art performance on multiple benchmark datasets.