FF-Net: Feature-Fusion-Based Network for Semantic Segmentation of 3D Plant Point Cloud

Semantic segmentation of 3D point clouds has played an important role in the field of plant phenotyping in recent years. However, existing methods need to down-sample the point cloud to a relatively small size when processing large-scale plant point clouds, which contain more than hundreds of thousands of points, which fails to take full advantage of the high-resolution of advanced scanning devices. To address this issue, we propose a feature-fusion-based method called FF-Net, which consists of two branches, namely the voxel-branch and the point-branch. In particular, the voxel-branch partitions a point cloud into voxels and then employs sparse 3D convolution to learn the context features, and the point-branch learns the point features within a voxel to preserve the detailed point information. Finally, an attention-based module was designed to fuse the two branch features to produce the final segmentation. We conducted extensive experiments on two large plant point clouds (maize and tomato), and the results showed that our method outperformed three commonly used models on both datasets and achieved the best mIoU of 80.95% on the maize dataset and 86.65% on the tomato dataset. Extensive cross-validation experiments were performed to evaluate the generalization ability of the models, and our method achieved promising segmentation results. In addition, the drawbacks of the proposed method were analyzed, and the directions for future works are given.

Automated summary

Semantic segmentation of 3D point clouds has been important in plant phenotyping. Existing methods down-sample point clouds when processing large-scale plant point clouds, failing to fully utilize high-resolution scanning devices. To address this, we propose FF-Net, a feature-fusion-based method with voxel and point branches. Our method outperformed three commonly used models on maize and tomato datasets, achieving the best mIoU of 80.95% and 86.65% respectively. Cross-validation experiments confirmed the generalization ability of our method.