Real-time dense stereo for intelligent vehicles

Stereo vision is an attractive passive sensing technique for obtaining three-dimensional (3-D) measurements. Recent hardware advances have given rise to a new class of real-time dense disparity estimation algorithms. This paper examines their suitability for intelligent vehicle (IV) applications. In order to gain a better understanding of the performance and the computational-cost tradeoff, the authors created a framework of real-time implementations. This consists of different methodical components based on single instruction multiple data (SIMD) techniques. Furthermore, the resulting algorithmic variations are compared with other publicly available algorithms. The authors argue that existing publicly available stereo data sets are not very suitable for the IV domain. Therefore, the authors' evaluation of stereo algorithms is based on novel realistically looking simulated data as well as real data from complex urban traffic scenes. In order to facilitate future benchmarks, all data used in this paper is made publicly available. The results from this study reveal that there is a considerable influence of scene conditions on the performance of all tested algorithms. Approaches that aim for (global) search optimization are more affected by this than other approaches. The best overall performance is achieved by the proposed multiple-window algorithm, which uses local matching and a left-right check for a robust error rejection. Timing results show that the simplest of the proposed SIMD variants are more than twice as fast than the most complex one. Nevertheless, the latter still achieves real-time processing speeds, while their average accuracy is at least equal to that of publicly available non-SIMD algorithms.