Journal
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING
Volume 60, Issue -, Pages -Publisher
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TGRS.2022.3197409
Keywords
Three-dimensional displays; Rendering (computer graphics); Remote sensing; Optimization; Image reconstruction; Neural networks; Estimation; Implicit neural network; multiplane images (MPIs); novel view synthesis; remote sensing (RS)
Categories
Funding
- National Natural Science Foundation of China [62125102]
Ask authors/readers for more resources
Novel view synthesis of remote sensing scenes plays a significant role in scene visualization, human-computer interaction, and various downstream applications. This article proposes a novel RS view synthesis method based on implicit neural representations, which outperforms previous state-of-the-art methods in terms of reconstruction accuracy, visual fidelity, and time efficiency.
Novel view synthesis of remote sensing (RS) scenes is of great significance for scene visualization, human-computer interaction, and various downstream applications. Despite the recent advances in computer graphics and photogrammetry technology, generating novel views is still challenging particularly for RS images due to its high complexity, view sparsity, and limited view-perspective variations. In this article, we propose a novel RS view synthesis method by leveraging the recent advances in implicit neural representations. Considering the overhead and far depth imaging of RS images, we represent the 3-D space by combining implicit multiplane images (ImMPI) representation and deep neural networks. The 3-D scene is reconstructed under a self-supervised optimization paradigm through a differentiable multiplane renderer with multiview input constraints. Images from any novel views thus can be freely rendered on the basis of the reconstructed model. As a by-product, the depth maps corresponding to the given viewpoint can be generated along with the rendering output. We refer to our method as ImMPIs. To further improve the view synthesis under sparse-view inputs, we explore the learning-based initialization of RS 3-D scenes and proposed a neural-network-based prior extractor to accelerate the optimization process. In addition, we propose a new dataset for RS novel view synthesis with multiview real-world Google Earth images. Extensive experiments demonstrate the superiority of the ImMPI over previous state-of-the-art methods in terms of reconstruction accuracy, visual fidelity, and time efficiency. Ablation experiments also suggest the effectiveness of our methodology design.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available