期刊
COMPUTER VISION AND IMAGE UNDERSTANDING
卷 219, 期 -, 页码 -出版社
ACADEMIC PRESS INC ELSEVIER SCIENCE
DOI: 10.1016/j.cviu.2022.103404
关键词
Cross-modal retrieval; Distributional matching
资金
- Visvesvaraya fellowship
This paper addresses the problem of learning a shared representation space for cross-modal retrieval. The authors propose the use of Discriminative Semantic Transitive Consistency to ensure correct classifications even after transferring the data points to another modality. They also enforce a traditional distance minimizing constraint and incorporate semantic cycle-consistency. The authors demonstrate better performance through empirical studies and provide qualitative results to support their proposals.
Cross-modal retrieval is generally performed by projecting and aligning the data from two different modalities onto a shared representation space. This shared space often also acts as a bridge for translating the modalities. We address the problem of learning such representation space by proposing and exploiting the property of Discriminative Semantic Transitive Consistency-ensuring that the data points are correctly classified even after being transferred to the other modality. Along with semantic transitive consistency, we also enforce the traditional distance minimizing constraint which makes the projections of the corresponding data points from both the modalities to come closer in the representation space. We analyze and compare the contribution of both the loss terms and their interaction, for the task. In addition, we incorporate semantic cycle-consistency for each of the modality. We empirically demonstrate better performance owing to the different components with clear ablation studies. We also provide qualitative results to support the proposals.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据