Dense 3D Reconstruction of Building Scenes by AI-Based Camera-Lidar Fusion and Odometry

Scanning is a key element for many use cases in the architectural, engineering, construction, and operation industry. It provides point clouds used for construction quality assurance, scan-to-building information modelling (BIM) workflows, and construction surveys. However, data acquisition using static laser scanners or photogrammetry methods is labor-intensive during scanning and postprocessing. Mobile scanners are conceptually the solution to this problem, given their potential to dramatically reduce onsite scanning effort and eliminate postprocessing work. However, current mobile mapping devices are limited to producing point clouds of relatively low resolution. In this paper, we propose a dense three-dimensional (3D) reconstruction pipeline for improving the resolution of point clouds, suitable for handheld scanners comprised of a color camera and a lidar. We fuse time-synchronized and spatially registered images and lidar sweeps using a spatial artificial intelligence (AI) method into dense scans of higher resolution, which are then used for progressive reconstruction. The novelty of our approach is that we first increase the precision and density of a bunch of individual lidar scans by inferring additional geometric constraints coming from predicted feature maps in the corresponding images. Then, we automatically register these scans together, thus reconstructing the scene progressively in an odometric manner. We built a prototypic scanner, implemented our reconstruction pipeline as a software package, and tested the whole in both indoor and outdoor case studies. The results showed that our method provided an overall noise reduction in point clouds by 11% and increased their density around six times.

Automated summary

In this paper, a dense 3D reconstruction pipeline is proposed to improve the resolution of point clouds captured by handheld scanners. Time-synchronized and spatially registered images and lidar sweeps are fused using spatial AI methods to generate higher resolution dense scans for progressive reconstruction. The results showed a reduction of 11% in overall point cloud noise and an increase in density by approximately six times.