High dynamic range 3D measurements based on space-time speckle correlation and color camera

Structured light (SL) based three-dimensional (3D) measurement struggles to estimate high dynamic range (HDR) scenes, where both high and low reflectivity parts exist simultaneously. This paper proposes a method through the joint design and optimization of hardware and algorithms, in which only four frames are required to realize the 3D reconstruction of HDR scenes. The height information of each sub-area in the scene under test can be encoded effectively by temporally projecting two sets of complementary speckle patterns onto target surface. To decode the corresponding patterns captured by the cameras, we design a stereo matching strategy consisting of space-time binary feature (ST-B1F) descriptor preliminary screening and zeromean normalized cross-correlation (ST-ZNCC) final retrieval. The ST-BIF descriptor based on neighborhood comparison is designed to describe the space-time relative intensity change of projected speckles. Besides the HDR adaptability, the ST-BIF descriptor can effectively improve the matching speed. In addition, the measurable dynamic range can be further improved by fusing all channel disparities as evaluated results, benefitting from the different response of R, G and B channels in color camera to monochromatic light. Experiments are conducted to demonstrate the feasibility of the proposed method. The results indicate that our method achieves the root mean square error 0.2516mm (vs. 1.0668 by commonly used ZNCC) and an average coverage rate up to 94.87% (vs. 93.35% by commonly used ZNCC). Furthermore, the experimental results show that the proposed method can achieve 3D reconstruction of HDR scenes including specular reflection region. (C) 2021 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

Automated summary

This paper proposes a method for 3D reconstruction of HDR scenes through joint design and optimization of hardware and algorithms, requiring only four frames. By temporally projecting two sets of complementary speckle patterns and using a stereo matching strategy, height information encoding and pattern decoding can be achieved effectively, improving matching speed and measurable dynamic range.