Journal
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
Volume 44, Issue 9, Pages 4674-4687Publisher
IEEE COMPUTER SOC
DOI: 10.1109/TPAMI.2021.3074370
Keywords
Kernel; Task analysis; Image segmentation; Semantics; Interpolation; Convolution; Object detection; Feature reassembly; object detection; instance segmentation; semantic segmentation; image inpainting
Funding
- SenseTime Group, Collaborative Research Grant, CUHK [TS1610626, TS1712093]
- General Research Fund of Hong Kong [14236516, 14203518]
- NTU SUG
- NTU NAP
- A*STAR
Ask authors/readers for more resources
CARAFE++ is a universal, lightweight, and highly effective operator for feature reassembly in convolutional networks. It aggregates contextual information within a large receptive field, generates adaptive kernels for instance-specific content-aware handling, and introduces little computational overhead. It consistently shows significant improvements in various tasks, making it a strong building block for modern deep networks.
Feature reassembly, i.e. feature downsampling and upsampling, is a key operation in a number of modern convolutional network architectures, e.g., residual networks and feature pyramids. Its design is critical for dense prediction tasks such as object detection and semantic/instance segmentation. In this work, we propose unified Content-Aware ReAssembly of FEatures (CARAFE++), a universal, lightweight, and highly effective operator to fulfill this goal. CARAFE++ has several appealing properties: (1) Unlike conventional methods such as pooling and interpolation that only exploit sub-pixel neighborhood, CARAFE++ aggregates contextual information within a large receptive field. (2) Instead of using a fixed kernel for all samples (e.g. convolution and deconvolution), CARAFE++ generates adaptive kernels on-the-fly to enable instance-specific content-aware handling. (3) CARAFE++ introduces little computational overhead and can be readily integrated into modern network architectures. We conduct comprehensive evaluations on standard benchmarks in object detection, instance/semantic segmentation, and image inpainting. CARAFE++ shows consistent and substantial gains on mainstream methods across all the tasks with negligible computational overhead. It shows great potential to serve as a strong building block for modern deep networks.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available