Depth Estimation Using an Infrared Dot Projector and an Infrared Color Stereo Camera

This paper proposes a method of estimating depth from two kinds of stereo images: color stereo images and infrared stereo images. An infrared dot pattern is projected on a scene by a projector so that infrared cameras can capture the scene textured by the dots and the depth can be estimated even where the surface is not textured. The cost volumes are calculated for the infrared and color stereo images for each frame and are extended in the time direction to define a spatiotemporal cost volume (st-cost volume). We also extend the cost volume filter in the time direction by modifying the cross-based local multipoint filter (CLMF) and applying it to the st-cost volumes in order to restrain flicker on the time-varying depth maps. To get a reliable cost volume, the infrared and color st-cost volumes are integrated into a single cost volume by selecting the cost of either the infrared or the color st-cost volumes according to the size of the adaptive kernel used for the CLMF. Then, a graphcut is executed on the cost volume in order to estimate the disparity robustly even when the baselines of the stereo cameras are set wide enough to ensure spatially high resolution in the depth direction and the shapes of blocks are deformed by the affine transformation. A 2D graphcut is executed on each scan line to reduce the processing time and memory consumption. We experimented with the proposed method using infrared color stereo data sets of scenes in the real world and evaluated its effectiveness by comparing it with other recent stereo matching methods and depth cameras.