Compound Projection Learning for Bridging Seen and Unseen Objects

Zero-shot Learning (ZSL) aims to transfer knowledge from seen image categories to unseen ones by leveraging semantic information. It is generally assumed that the seen and unseen objects share a common semantic space. Most of existing ZSL methods focus on how to connect the visual space and the semantic space. However, since there are some visual distribution differences between seen and unseen objects, the projection function learned by those seen classes is biased when transferring knowledge to unseen classes. We argue that, although the unseen objects are class-agnostic, the visual distribution information of unseen samples can be generated by exploiting semantic features. In this paper, we propose a Compound Projection Learning (CPL) model to transfer knowledge from seen to unseen objects by exploiting the information of both seen and class-agnostic samples. With the projected semantic representation by CPL, effective constraints such as projection loss and semantic reconstruction loss can be explored for seen and unseen objects, respectively, such that the semantic ambiguity across seen and unseen objects is reduced. Additionally, we utilize a similarity network to further explore the inter-class relationship by employing the labels and the similarities between seen and unseen classes. Extensive experiments on ZSL benchmark datasets show the effectiveness of our proposed approach.

Automated summary

Zero-shot Learning aims to transfer knowledge from seen image categories to unseen ones by leveraging semantic information. However, existing methods often suffer from biased learned projection functions due to visual distribution differences between seen and unseen objects. In this paper, we propose a Compound Projection Learning (CPL) model that exploits both seen and class-agnostic samples to reduce semantic ambiguity and improve knowledge transfer. We also utilize a similarity network to explore inter-class relationships. Experimental results on ZSL benchmark datasets demonstrate the effectiveness of our approach.