A zero-shot learning framework via cluster-prototype matching

Given the descriptions of classes, Zero-Shot Learning (ZSL) aims to recognize unseen samples by learning a projection between the visual features of samples and the semantic descriptions (prototypes) of classes from seen data. However, due to the inherent distribution gap between seen and unseen domains, the learned projection is generally biased to seen classes and may produce misleading relationships between unseen samples and prototypes (sample-prototype relationship). To tackle this problem, we propose a Cluster-Prototype Matching (CPM) framework which exploits the distribution information of samples to explore the cluster structure of samples and then use the robust cluster-prototype relationship to correct the biased sample-prototype relationship. Specifically, we first use an iterative cluster generation mod-ule to identify the underlying cluster structure of samples based on their embedding features, which are acquired via a basic ZSL model. Then each identified cluster will be matched with a specific class pro-totype through the Kuhn-Munkres algorithm, based on which we can export a sharp cluster-prototype similarity. Finally, the cluster-prototype similarity is combined with the sample-prototype similarity to determine the class labels of test samples. We apply CPM to five well-established ZSL methods and the experimental results show that CPM can significantly improve the performance of basic models and en-able them achieve or beyond the state-of-the-art. (c) 2021 Elsevier Ltd. All rights reserved.

Automated summary

The paper introduces a Cluster-Prototype Matching (CPM) framework that utilizes the distribution information of samples to correct the biased relationships between seen and unseen classes, improving the performance and effectiveness of zero-shot learning.