期刊
IEEE TRANSACTIONS ON MULTIMEDIA
卷 19, 期 6, 页码 1220-1233出版社
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TMM.2016.2646219
关键词
Cross-modal retrieval; documents and images; multimedia
资金
- National Basic Research Program of China (973 Program [2015CB351800, 2012CB316400]
- National Natural Science Foundation of China [61572465, 61332016, 61429201, 61620106009, U1636214, 61303153]
- Key Research Program of Frontier Sciences, Chinese Academy of Sciences [QYZDJ-SSW-SYS013]
- ARO [W911NF-15-1-0290]
- Faculty Research Gift Awards by the NEC Laboratories of America and Blippar
This paper proposes a novel method for cross-modal retrieval. In addition to the traditional vector (text)-to-vector (image) framework, we adopt a matrix (text)-to-matrix (image) framework to faithfully characterize the structures of different feature spaces. Moreover, we propose a novel metric learning framework to learn a discriminative structured subspace, in which the underlying data distribution is preserved for ensuring a desirablemetric. Concretely, there are three steps for the proposed method. First, the multiorder statistics are used to represent images and texts for enriching the feature information. We jointly use the covariance (second-order), mean (first-order), and bags of visual (textual) features (zeroth-order) to characterize each image and text. Second, considering that the heterogeneous covariance matrices lie on the different Riemannian manifolds and the other features on the different Euclidean spaces, respectively, we propose a unified metric learning framework integrating multiple distance metrics, one for each order statistical feature. This framework preserves the underlying data distribution and exploits complementary information for better matching heterogeneous data. Finally, the similarity between the different modalities can be measured by transforming the multiorder statistical features to the common subspace. The performance of the proposed method over the previous methods has been demonstrated through the experiments on two public datasets.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据