Stereo vision architecture for heterogeneous systems-on-chip

Stereo vision is a crucial operation in state-of-the-art computer vision applications. Several efficient algorithms have been recently proposed either to increase the quality of the produced disparity maps or to reach higher computational speed, or both. Among them, hardware-oriented algorithms are desirable when the main objective is including stereo vision in portable consumer electronic and multimedia systems. Modern FPGA-based platforms allow all-programmable heterogeneous embedded systems to be realized as SoCs and they are certainly appropriate also to implement stereo vision. This paper proposes a novel stereo vision algorithm and a specific implementation suitable for heterogeneous SoC FPGA-based embedded systems. A complete embedded system design is also demonstrated using the Xilinx Zynq-7000 SoCs device family. The novel algorithm has been characterized in terms of accuracy by referring to the benchmark sets Middlebury and Kitti. When 640 x 480 8-bit greyscale stereo pairs are processed, the fastest prototype, realized within the XC7Z020 and the XC7Z045 device, respectively, exhibits an 81 and 101 fps frame rate. Conversely, the cheapest implementation occupies only 52691 LUTs, 59715 FFs, 93 DSPs, 40 BRAM18k and 6 BRAM36k memory blocks. In comparison to several counterparts existing in literature, the novel algorithm can achieve higher accuracies, and makes the proposed complete system design able to reach the most favourable accuracy/performance trade-off.