Image stitching by disparity-guided multi-plane alignment

Image stitching aims to warp and align two or more images with overlapping areas to generate a panorama with a larger field of view. Due to the wide baseline or the abrupt depth changes caused by the foreground objects, the phenomenon that adjacent pixels or regions in one image are not adjacent in another image happens. It is difficult to avoid severe parallax artifacts and get good alignment results when stitching such images. In this paper, focusing on the images with large parallax, we design a multi-plane alignment algorithm guided by the disparity map. The concept of average tolerable parallax is proposed to help distinguish one background plane and multiple foreground objects from the image, which are robust to small parallax. In order to obtain the reliable disparity map of the image pairs got from any camera with unknown camera parameters, we propose an optimal homography estimation method based on the relative projection biases. This helps to satisfy the common epipolar line constraint when applying stereo matching modules. Experimental results demonstrate that our algorithm provides accurate stitching results on images with large parallax, and outperforms other existing state-of-the-art methods both qualitatively and quantitatively. (C) 2022 The Author(s). Published by Elsevier B.V.

Automated summary

Image stitching aims to generate a panorama with a larger field of view by warping and aligning two or more images with overlapping areas. This paper proposes a multi-plane alignment algorithm guided by the disparity map for images with large parallax. The concept of average tolerable parallax is introduced to distinguish background planes and foreground objects. An optimal homography estimation method based on relative projection biases is proposed to obtain reliable disparity maps. Experimental results demonstrate the accuracy and superiority of the algorithm compared to existing methods.