Integrating Deep Semantic Segmentation Into 3-D Point Cloud Registration

Point cloud registration is the task of aligning 3D scans of the same environment captured from different poses. When semantic information is available for the points, it can be used as a prior in the search for correspondences to improve registration. Semantic-assisted Normal Distributions Transform (SE-NDT) is a new registration algorithm that reduces the complexity of the problem by using the semantic information to partition the point cloud into a set of normal distributions, which are then registered separately. In this letter we extend the NDT registration pipeline by using PointNet, a deep neural network for segmentation and classification of point clouds, to learn and predict per-point semantic labels. We also present the Iterative Closest Point (ICP) equivalent of the algorithm, a special case of Multichannel Generalized ICP. We evaluate the performance of SE-NDT against the state of the art in point cloud registration on the publicly available classification data set Semantic3d.net. We also test the trained classifier and algorithms on dynamic scenes, using a sequence from the public dataset KITTI. The experiments demonstrate the improvement of the registration in terms of robustness, precision and speed, across a range of initial registration errors, thanks to the inclusion of semantic information.