Multi-View 3D Shape Recognition via Correspondence-Aware Deep Learning

In recent years, multi-view learning has emerged as a promising approach for 3D shape recognition, which identifies a 3D shape based on its 2D views taken from different viewpoints. Usually, the correspondences inside a view or across different views encode the spatial arrangement of object parts and the symmetry of the object, which provide useful geometric cues for recognition. However, such view correspondences have not been explicitly and fully exploited in existing work. In this paper, we propose a correspondence-aware representation (CAR) module, which explicitly finds potential intra-view correspondences and cross-view correspondences via kNN search in semantic space and then aggregates the shape features from the correspondences via learned transforms. Particularly, the spatial relations of correspondences in terms of their viewpoint positions and intra-view locations are taken into account for learning correspondence-aware features. Incorporating the CAR module into a ResNet-18 backbone, we propose an effective deep model called CAR-Net for 3D shape classification and retrieval. Extensive experiments have demonstrated the effectiveness of the CAR module as well as the excellent performance of the CAR-Net.

Automated summary

In recent years, multi-view learning has become a promising approach for 3D shape recognition by identifying shapes based on 2D views from different angles. This paper proposes a correspondence-aware representation (CAR) module that finds potential intra-view and cross-view correspondences through kNN search in semantic space and aggregates shape features via learned transforms. Incorporating the CAR module into a ResNet-18 backbone, an effective deep model called CAR-Net is introduced for 3D shape classification and retrieval, demonstrating the effectiveness and excellent performance of the CAR module.