Dense view synthesis for three-dimensional light-field displays based on position-guiding convolutional neural network

Dense view synthesis from sparse views is a fundamental and significant research field, which has attracted considerable attention in Three-Dimensional (3D) light-field displays. As we known, capturing stereo views from binocular cameras is always a fast and convenient way in engineering applications. Synthesizing dense high quality views from stereo views, however, is still a challenging problem. Here, to address this problem, a novel method of dense-view synthesis based on a position-guiding convolutional neural network (CNN) is proposed, which only uses left and right views as input when synthesizing novel virtual views. To handle the occluded areas on left-right views, the depth map of a middle view is exploited with a depth estimation CNN. Furthermore, a normalized position factor is proposed to adjust the depth information by scaling operation for guiding the dense view synthesis in a view-rectifying CNN. The experiments show that our approach could synthesize high-quality dense views and the SSIM is above 0.94. Besides, the synthesized dense views are presented on our light-field device and perform continuous and reasonable occlusion relations. We think our work can find wide applications of 3D light-field displays in the future.

Automated summary

The article proposes a novel method based on a position-guiding convolutional neural network for dense view synthesis from sparse views, utilizing depth maps from left, right, and middle views to achieve high-quality dense view synthesis. Experimental results demonstrate that the approach can synthesize high-quality dense views, with SSIM above 0.94, and present continuous and reasonable occlusion relations on a light-field device, showing promising applications in 3D light-field displays.