GO-Surf: Neural Feature Grid Optimization for Fast, High-Fidelity RGB-D Surface Reconstruction

We present GO-Surf, a direct feature grid optimization method for accurate and fast surface reconstruction from RGB-D sequences. We model the underlying scene with a learned hierarchical feature voxel grid that encapsulates multi-level geometric and appearance local information. Feature vectors are directly optimized such that after being tri-linearly interpolated, decoded by two shallow MLPs into signed distance and radiance values, and rendered via volume rendering, the discrepancy between synthesized and observed RGB/depth values is minimized. Our supervision signals - RGB, depth and approximate SDF - can be obtained directly from input images without any need for fusion or post-processing. We formulate a novel SDF gradient regularization term that encourages surface smoothness and hole filling while maintaining high frequency details. GO-Surf can optimize sequences of 1-2K frames in 15-45 minutes, a speedup of x60 over NeuralRGB-D [1], the most related approach based on an MLP representation, while maintaining on par performance on standard benchmarks. Project page: https://jingwenwang95.github. io/go_surf.

Automated summary

GO-Surf is a direct feature grid optimization method for accurate and fast surface reconstruction from RGB-D sequences. It models the scene with a learned hierarchical feature voxel grid and optimizes feature vectors to minimize the discrepancy between synthesized and observed RGB/depth values. GO-Surf also introduces a novel SDF gradient regularization term to encourage surface smoothness and hole filling while maintaining high frequency details. It achieves a significant speedup over the most related MLP-based approach while maintaining comparable performance on standard benchmarks.