KD-PatchMatch: A Self-Supervised Training Learning-Based PatchMatch

Traditional learning-based multi-view stereo (MVS) methods usually need to find the correct depth value from a large number of depth candidates, which leads to huge memory consumption and slow inference. To address these problems, we propose a probabilistic depth sampling in the learning-based PatchMatch framework, i.e., sampling a small number of depth candidates from a single-view probability distribution, which achieves the purpose of saving computational resources. Furthermore, to overcome the difficulty of obtaining ground-truth depth for outdoor large-scale scenes, we also propose a self-supervised training pipeline based on knowledge distillation, which involves self-supervised teacher training and student training based on knowledge distillation. Extensive experiments show that our approach outperforms other recent learning-based MVS methods on DTU, Tanks and Temples, and ETH3D datasets.

Automated summary

Traditional learning-based multi-view stereo (MVS) methods often face issues of high memory consumption and slow inference due to searching correct depth values from a large number of candidates. To solve these problems, we propose a probabilistic depth sampling technique that selects a small number of candidates from a probability distribution, thereby saving computational resources. Additionally, we introduce a self-supervised training pipeline based on knowledge distillation to handle the challenge of obtaining ground-truth depth for outdoor large-scale scenes. Extensive experiments demonstrate that our approach surpasses other recent learning-based MVS methods on multiple datasets.