Few-shot learning-based RGB-D salient object detection: A case study

RGB-D salient object detection (SOD) aims at detecting general attention-grabbing objects from paired RGB and depth image inputs, and recently has attracted increasing research attention. Despite that many advanced RGB-D SOD models are proposed, almost all of them focus on developing models in a fully supervised manner with a small training dataset that typically has only hundreds of RGB-D samples. This may inevitably incur poor generalizability of these models when being applied to real-world scenar-ios and applications. To narrow such a gap, we make the first attempt of treating RGB-D SOD as a few -shot learning (FSL) problem, and improve it by introducing extra prior knowledge from a closely related task, i.e., RGB SOD. Inspired by the general taxonomy of FSL techniques, we investigate from two perspec-tives, namely model and data, of transferring additional knowledge from the RGB SOD dataset to enhance RGB-D SOD performance. For the former, we employ multi-task learning with parameter sharing to con-strain the model space, whereas for the latter, we propose to generate the depth from RGB by using an off-the-shelf depth estimator. Representative middle-fusion and late-fusion models are trialed and vali-dated under such a FSL setup. Our experimental results and analyses confirm the feasibility of promoting RGB-D SOD via FSL techniques, while comparative study on different FSL techniques and detection strate-gies is conducted. We hope this work can serve as a catalyst for bringing RGB-D saliency detection into real applications, as well as for inspiring future works that apply few-shot learning to saliency detection and other multi-modal detection tasks.(c) 2022 Elsevier B.V. All rights reserved.

Automated summary

RGB-D salient object detection is an important research task, but existing models often suffer from poor generalizability. This paper treats RGB-D salient object detection as a few-shot learning problem and introduces prior knowledge from a closely related task, RGB salient object detection, to enhance performance. Experimental results validate the feasibility of using few-shot learning techniques to improve RGB-D salient object detection.