Contrastive Adversarial Domain Adaptation Networks for Speaker Recognition

Domain adaptation aims to reduce the mismatch between the source and target domains. A domain adversarial network (DAN) has been recently proposed to incorporate adversarial learning into deep neural networks to create a domain-invariant space. However, DAN's major drawback is that it is difficult to find the domain-invariant space by using a single feature extractor. In this article, we propose to split the feature extractor into two contrastive branches, with one branch delegating for the class-dependence in the latent space and another branch focusing on domain-invariance. The feature extractor achieves these contrastive goals by sharing the first and last hidden layers but possessing decoupled branches in the middle hidden layers. For encouraging the feature extractor to produce class-discriminative embedded features, the label predictor is adversarially trained to produce equal posterior probabilities across all of the outputs instead of producing one-hot outputs. We refer to the resulting domain adaptation network as contrastive adversarial domain adaptation network (CADAN). We evaluated the embedded features' domain-invariance via a series of speaker identification experiments under both clean and noisy conditions. Results demonstrate that the embedded features produced by CADAN lead to a 33% improvement in speaker identification accuracy compared with the conventional DAN.

Automated summary

In this article, a contrastive adversarial domain adaptation network (CADAN) is proposed to address the difficulty of finding domain-invariant space in a single feature extractor. By splitting the feature extractor into two branches, CADAN achieves class-dependence and domain-invariance simultaneously. Evaluation experiments demonstrate that CADAN outperforms conventional methods in terms of speaker identification accuracy.