Beyond Photometric Consistency: Geometry-Based Occlusion-Aware Unsupervised Light Field Disparity Estimation

Although learning-based light field disparity estimation has achieved great progress in the most recent years, the performance of unsupervised light field learning is still hindered by occlusions and noises. By analyzing the overall strategy underlying the unsupervised methodology and the light field geometry implied in epipolar plane images (EPIs), we look beyond the photometric consistency assumption, and design an occlusion-aware unsupervised framework to deal with the situations of photometric consistency conflict. Specifically, we present a geometry-based light field occlusion modeling, which predicts a group of visibility masks and occlusion maps, respectively, by forward warping and backward EPI-line tracing. In order to learn better the noise-and occlusion-invariant representations of the light field, we propose two occlusion-aware unsupervised losses: occlusion-aware SSIM and statistics-based EPI loss. Experiment results demonstrate that our method can improve the estimation accuracy of light field depth over the occluded and noisy regions, and preserve the occlusion boundaries better.

Automated summary

Although learning-based light field disparity estimation has made great progress recently, unsupervised light field learning still faces challenges from occlusions and noises. By analyzing the unsupervised methodology and the light field geometry implied in epipolar plane images (EPIs), a new occlusion-aware unsupervised framework is proposed to address the issue of photometric consistency conflict. This framework includes a geometry-based light field occlusion model and two occlusion-aware unsupervised losses to improve estimation accuracy and preserve occlusion boundaries.