A joint framework for mining discriminative and frequent visual representation

Discovering visual representation in an image category is a challenging issue, because the visual repre-sentation should not only be discriminative but also frequently appears in these images. Previous studies have proposed many solutions, however, all of them separately optimized the discrimination and fre-quency, which consequently makes the solutions sub-optimal. We propose a method to discover the jointly discriminative and frequent visual representation to address this issue, named as JDFR. To ensure discrimination, JDFR employs a classification task with cross-entropy loss. To achieve frequency, we design a novel similarity concentration (SC) loss to concentrate on the samples with the same represen-tation and pull them closer in the feature space, and then mine the frequent visual representations. Moreover, we utilize an attention module to locate the representative region in the image. Extensive experiments on five benchmark datasets (Place365-20, Travel, VOC2012-10, ImageNet-100, and iNaturalist-100) show that the discovered visual representations have better discrimination and fre-quency than ones mined by the state-of-the-art (SOTA) method with average improvements of 5.37% on accuracy and 3.06% on frequency.(c) 2022 Elsevier B.V. All rights reserved.

Automated summary

Discovering visually discriminative and frequent representations in image categories is a challenging issue. Previous studies optimized discrimination and frequency separately, resulting in sub-optimal solutions. In this paper, we propose a method named JDFR that discovers joint discriminative and frequent visual representations. By employing a classification task with cross-entropy loss and a similarity concentration loss, JDFR ensures discrimination and frequency, and utilizes an attention module to locate representative regions in images.