期刊
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
卷 45, 期 8, 页码 9486-9503出版社
IEEE COMPUTER SOC
DOI: 10.1109/TPAMI.2023.3242709
关键词
View synthesis; light field; deep learning; image warping; depth/disparity
This paper proposes a content-aware warping method that adaptsively learns the interpolation weights for pixels from their contextual information via a lightweight neural network. Based on this learnable warping module, a new end-to-end learning-based framework is proposed for novel view synthesis, which includes two additional modules to address occlusion and spatial correlation issues. Experimental results demonstrate that the proposed method outperforms state-of-the-art methods both quantitatively and visually.
Existing image-based rendering methods usually adopt depth-based image warping operation to synthesize novel views. In this paper, we reason the essential limitations of the traditional warping operation to be the limited neighborhood and only distance-based interpolation weights. To this end, we propose content-aware warping, which adaptively learns the interpolation weights for pixels of a relatively large neighborhood from their contextual information via a lightweight neural network. Based on this learnable warping module, we propose a new end-to-end learningbased framework for novel view synthesis from a set of input source views, in which two additional modules, namely confidence-based blending and feature-assistant spatial refinement, are naturally proposed to handle the occlusion issue and capture the spatial correlation among pixels of the synthesized view, respectively. Besides, we also propose a weight-smoothness loss term to regularize the network. Experimental results on light field datasets with wide baselines and multi-view datasets show that the proposed method significantly outperforms state-of-the-art methods both quantitatively and visually. The source code is publicly available at https://github.com/MantangGuo/CW4VS.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据