An attention-based and deep sparse priori cascade multi-view stereo network for 3D reconstruction

Recently 3D reconstruction has received extensive attention and deep research due to the con-tinuous development of the field of computer vision. Nonetheless, the learning-based methods for 3D reconstruction are not robust enough, resulting in the reconstruction model having many occlusions and outliers. In this study, we aim to improve feature matching correlation, aggregate global contextual information, and enhance the robustness of depth estimation to improve the quality of the reconstruction. We propose an attention-based deep sparse priori cascade multi-view stereo network, ADS-MVSNet. Firstly, we propose a feature extraction module based on the attention mechanism to obtain the regions of interest in the input scene. Secondly, we propose a depth sparse prior strategy module to estimate the depth map of the input scene more accurately. It is followed by refinement of the initial depth map using a coarse-to-fine method to improve the accuracy of point cloud reconstruction. These two modules are lightweight and effective, improving the robustness of depth map estimation. We conduct experiments on three common datasets (DTU, ETH3D, Tanks & Temples) and a dataset of a real scene created by us. The experimental results show that ADS-MVSNet performs better in reconstruction quality compared to classical methods.(c) 2023 Elsevier Ltd. All rights reserved.

Automated summary

This study aims to improve the quality of 3D reconstruction by addressing the lack of robustness in learning-based methods for depth estimation. A attention-based deep sparse priori cascade multi-view stereo network, ADS-MVSNet, is proposed. It utilizes a feature extraction module based on the attention mechanism and a depth sparse prior strategy module to accurately estimate the depth map and refine it using a coarse-to-fine method for better point cloud reconstruction.