Deep Corner

Recent studies have shown promising results on joint learning of local feature detectors and descriptors. To address the lack of ground-truth keypoint supervision, previous methods mainly inject appropriate knowledge about keypoint attributes into the network to facilitate model learning. In this paper, inspired by traditional corner detectors, we develop an end-to-end deep network, named Deep Corner, which adds a local similarity-based keypoint measure into a plain convolutional network. Deep Corner enables finding reliable keypoints and thus benefits the learning of the distinctive descriptors. Moreover, to improve keypoint localization, we first study previous multi-level keypoint detection strategies and then develop a multi-level U-Net architecture, where the similarity of features at multiple levels can be exploited effectively. Finally, to improve the invariance of descriptors, we propose a feature self-transformation operation, which transforms the learned features adaptively according to the specific local information. The experimental results on several tasks and comprehensive ablation studies demonstrate the effectiveness of our method and the involved components.

Automated summary

Recent studies have achieved promising results by jointly learning local feature detectors and descriptors. To overcome the lack of ground-truth keypoint supervision, previous methods have incorporated relevant knowledge about keypoint attributes into the network for enhanced model learning. This paper presents Deep Corner, an end-to-end deep network that combines a local similarity-based keypoint measure with a plain convolutional network, inspired by traditional corner detectors. The proposed method yields reliable keypoints, facilitate the learning of distinctive descriptors. Additionally, the paper introduces a multi-level U-Net architecture and a feature self-transformation operation to further improve keypoint localization and descriptor invariance.