Synthetic Aperture Based on Plenoptic Camera for Seeing Through Occlusions

De-occlusion is a classical challenge in computer vision to many research fields. In this paper, a synthetic aperture algorithm is proposed to see through occlusions based on plenoptic camera (PLC). The sheltered object is located onto the zero-disparity plane by deriving the geometric relationship between the focus plane and the object plane using the virtual camera array structure of PLC. Pixel selection relying on depth and motion clustering is proposed to pick out the pixels coming from sheltered object. Finally, a synthetic aperture image (SAI) is obtained by averaging the selected pixels so that the sheltered objects can be seen. Compared with the existing work, the proposed method provides much better visual quality in achieving clearer sheltered objects and much more blurry occlusions.