4.7 Article

ASK: Adaptively Selecting Key Local Features for RGB-D Scene Recognition

Journal

IEEE TRANSACTIONS ON IMAGE PROCESSING
Volume 30, Issue -, Pages 2722-2733

Publisher

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TIP.2021.3053459

Keywords

Feature extraction; Image recognition; Object detection; Training; Correlation; Layout; Convolution; RGB-D recognition; local feature selection; multi-modal feature learning

Funding

  1. National Key Research and Development Project [2020YFB 2103902]
  2. National Natural Science Foundation of China [61632018, 61825603, U1801262, 61871470]

Ask authors/readers for more resources

An efficient framework for RGB-D scene recognition is proposed in this article, which adaptively selects important local features to capture the spatial variability of scene images. By designing a differentiable local feature selection (DLFS) module, key local scene-related features can be extracted from spatially-correlated multi-modal RGB-D features. By concatenating local-orderless and global-structured multi-modal features, the proposed framework achieves state-of-the-art performance on public RGB-D scene recognition datasets.
Indoor scene images usually contain scattered objects and various scene layouts, which make RGB-D scene classification a challenging task. Existing methods still have limitations for classifying scene images with great spatial variability. Thus, how to extract local patch-level features effectively using only image label is still an open problem for RGB-D scene recognition. In this article, we propose an efficient framework for RGB-D scene recognition, which adaptively selects important local features to capture the great spatial variability of scene images. Specifically, we design a differentiable local feature selection (DLFS) module, which can extract the appropriate number of key local scene-related features. Discriminative local theme-level and object-level representations can be selected with DLFS module from the spatially-correlated multi-modal RGB-D features. We take advantage of the correlation between RGB and depth modalities to provide more cues for selecting local features. To ensure that discriminative local features are selected, the variational mutual information maximization loss is proposed. Additionally, the DLFS module can be easily extended to select local features of different scales. By concatenating the local-orderless and global-structured multi-modal features, the proposed framework can achieve state-of-the-art performance on public RGB-D scene recognition datasets.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.7
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

No Data Available
No Data Available