期刊
WORLD WIDE WEB-INTERNET AND WEB INFORMATION SYSTEMS
卷 22, 期 2, 页码 657-672出版社
SPRINGER
DOI: 10.1007/s11280-018-0541-x
关键词
Cross-modal retrieval; Adversarial learning; Metric learning
资金
- NSFC [61602089, 61673088, 61502080]
- 111 Project [B17008]
- Fundamental Research Funds for Central Universities [ZYGX2016KYQD114]
- LEADER of MEXT-Japan [16809746]
- Telecommunications Foundation
- REDAS
- SCAT
Cross-modal retrieval has become a highlighted research topic, to provide flexible retrieval experience across multimedia data such as image, video, text and audio. The core of existing cross-modal retrieval approaches is to narrow down the gap between different modalities either by finding a maximally correlated embedding space. Recently, researchers leverage Deep Neural Network (DNN) to learn nonlinear transformations for each modality to obtain transformed features in a common subspace where cross-modal matching can be performed. However, the statistical characteristics of the original features for each modality are not explicitly preserved in the learned subspace. Inspired by recent advances in adversarial learning, we propose a novel Deep Adversarial Metric Learning approach, termed DAML for cross-modal retrieval. DAML nonlinearly maps labeled data pairs of different modalities into a shared latent feature subspace, under which the intra-class variation is minimized and the inter-class variation is maximized, and the difference of each data pair captured from two modalities of the same class is minimized, respectively. In addition to maximizing the correlations between modalities, we add an additional regularization by introducing adversarial learning. In particular, we introduce a modality classifier to predict the modality of a transformed feature, which ensures that the transformed features are also statistically indistinguishable. Experiments on three popular multimodal datasets show that DAML achieves superior performance compared to several state of the art cross-modal retrieval methods.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据