Multi-hierarchy feature extraction and multi-step cost aggregation for stereo matching

Compared with the traditional hand-crafted feature based methods, learning-based stereo matching methods have made great progress in matching accuracy. However, current CNN-based stereo matching methods usually require a lot of time and memory consumption. It is very difficult to achieve the good balance between disparity estimation accuracy and inference speed that is significant to the application in real scenarios. To this end, we propose a accurate and fast stereo matching network (named MMNet), which contains two key modules of Multi-hierarchy feature extraction and Multi-step cost aggregation. In order to achieve a good trade-off between better disparity estimation and faster inference speed, a lightweight multi-hierarchy feature extractor is first proposed. This module obtains reliable feature information of different scales through three stable scale hierarchy branches, and outputs multi-step feature flows containing multi-scale fusion information at each step of the highest scale hierarchy branch. Moreover, we also propose a multi-step cost aggregation scheme, which uses shallow features to guide cost aggregation for ensuring a better aggregation effect with a small number of 3D convolutions. The experimental results on SceneFlow, KITTI 2012 and KITTI 2015 datasets show that our proposed network achieves extremely competitive disparity estimation accuracy with fast inference speed. (C) 2021 Elsevier B.V. All rights reserved.

Automated summary

Compared with traditional hand-crafted feature based methods, learning-based stereo matching methods have made significant progress in matching accuracy. However, current CNN-based methods often require a substantial amount of time and memory consumption. To address this issue, we propose an accurate and fast stereo matching network that incorporates multi-hierarchy feature extraction and multi-step cost aggregation. Experimental results demonstrate that our network achieves highly competitive disparity estimation accuracy with fast inference speed.