Incorporating Co-Visibility Reasoning Into Surface Depth Measurement

Multiview stereo (MVS) aims to measure the precise surface depth of a scene from observations at multiple photography angles and then densely reconstruct its 3-D geometry information. Learning-based MVS approaches have been dominantly popular for their robustness to low texture areas and non-Lambertian surfaces. However, most existing methods focus on estimating depth maps for input images by constructing global cost volumes and designing ingenious yet large variance-based 3-D-CNNs for cost volume regularization. Such approaches ignore the co-visible relationship embedded in multiple views, resulting in heavy computation, erroneous cost aggregation from invisible views, and finally inaccurate 3-D reconstruction results. In this article, we propose a co-visibility reasoning MVS network (CR-MVSNet) to explore the co-visible relationships hidden in multiple views for reliable multiview similarity measurement and efficient reconstruction. Precisely, the proposed co-visibility reasoning cost aggregation (CRCA) module includes the adaptive intercost volume aggregation via mining the uncertainty of co-visibility relationships in multiple views and the adaptive intracost volume aggregation by exploiting spatial contextual information. Moreover, the cost volumes are constructed via the proposed global-to-patch manner to speed up computation. Experimental results show that our approach achieves the best overall performance on the DTU, Tanks and Temples, and ETH3D-test datasets over recent state-of-the-art MVS algorithms. The consistently favorable results on three datasets with completely different depth ranges proved the superiority and generalizability of CR-MVSNet.

Automated summary

This article proposes a multiview stereo network called CR-MVSNet, which utilizes co-visibility reasoning to achieve reliable multiview similarity measurement and efficient reconstruction. The network outperforms state-of-the-art MVS algorithms on multiple datasets.