Journal
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING
Volume 60, Issue -, Pages -Publisher
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TGRS.2021.3066432
Keywords
Feature extraction; Optical distortion; Image matching; Synthetic aperture radar; Optical sensors; Optical imaging; Nonlinear optics; Attention mechanism; convolutional neural network (CNN); deep learning; image matching; optical image; spatial pyramid pooling; synthetic aperture radar (SAR)
Categories
Funding
- National Key Research and Development Program of China [2017YFB0504202, 2018YFB0504801]
- National Natural Science Foundation of China [41771385, 41801267]
- Fundamental Research Funds for the Central Universities [2042020kf0014]
Ask authors/readers for more resources
The article proposes a novel image-based matching method MAP-Net for SAR to optical images, which utilizes spatial pyramid aggregated pooling (SPAP) and an attention mechanism to extract key features for image matching. Experimental results demonstrate that MAP-Net outperforms current state-of-the-art image matching methods for SAR to optical images.
The complementarity of synthetic aperture radar (SAR) and optical images allows remote sensing observations to ``see'' unprecedented discoveries. Image matching plays a fundamental role in the fusion and application of SAR and optical images. However, both the geometric imaging pattern and the physical radiation mechanism of these two sensors are significantly different, so that the images show complex geometric distortion and nonlinear radiation differences. This phenomenon brings great challenges to image matching, which neither the handcrafted descriptors nor the deep learning-based methods have adequately addressed. In this article, a novel image-based matching method for SAR to optical images via an image-based convolutional network with spatial pyramid aggregated pooling (SPAP) and an attention mechanism is proposed, namely MAP-Net. The original image is embedded through the convolutional neural network to generate the feature map. Through the information extraction and abstraction of the original imagery, the embedded features containing the high-level semantic information are more robust to the geometric distortion and radiation variation among the different modal images, which is beneficial to the matching of cross-modal images. The adoption of the SPAP module makes the network more capable of integrating global and local contextual information. The attention block weights the dense features generated from the network to extract the key features that are invariant, distinguishable, repeatable, and suitable for the image matching task. In the experiments, five sets of multisource and multiresolution SAR and optical images with wide and varied ground coverage were used to evaluate the accuracy of MAP-Net, compared to both handcrafted and deep learning-based methods. The experimental results show that the MAP-Net method is superior to the current state-of-the-art image matching methods for SAR to optical images.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available