Panicle-3D: A low-cost 3D-modeling method for rice panicles based on deep learning, shape from silhouette, and supervoxel clustering

Self-occlusions are common in rice canopy images and strongly influence the calculation accuracies of panicle traits. Such interference can be largely eliminated if panicles are phenotyped at the 3D level. Research on 3D panicle phenotyping has been limited. Given that existing 3D modeling techniques do not focus on specified parts of a target object, an efficient method for panicle modeling of large numbers of rice plants is lacking. This paper presents an automatic and nondestructive method for 3D panicle modeling. The proposed method integrates shoot rice reconstruction with shape from silhouette, 2D pan-icle segmentation with a deep convolutional neural network, and 3D panicle segmentation with ray trac-ing and supervoxel clustering. A multiview imaging system was built to acquire image sequences of rice canopies with an efficiency of approximately 4 min per rice plant.The execution time of panicle modeling per rice plant using 90 images was approximately 26 min. The outputs of the algorithm for a single rice plant are a shoot rice model, surface shoot rice model, panicle model, and surface panicle model, all rep-resented by a list of spatial coordinates. The efficiency and performance were evaluated and compared with the classical structure-from-motion algorithm. The results demonstrated that the proposed method is well qualified to recover the 3D shapes of rice panicles from multiview images and is readily adaptable to rice plants of diverse accessions and growth stages. The proposed algorithm is superior to the structure-from-motion method in terms of texture preservation and computational efficiency. The sam-ple images and implementation of the algorithm are available online. This automatic, cost-efficient, and nondestructive method of 3D panicle modeling may be applied to high-throughput 3D phenotyping of large rice populations.(c) 2022 2022 Crop Science Society of China and Institute of Crop Science, CAAS. Production and hosting by Elsevier B.V. on behalf of KeAi Communications Co., Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

Automated summary

This research presents an automatic and nondestructive method for 3D panicle modeling in rice plants. The method integrates various techniques such as shoot rice reconstruction, shape from silhouette, deep convolutional neural network, ray tracing, and supervoxel clustering. It demonstrates high efficiency and performance in recovering the 3D shapes of rice panicles from multiview images, and is adaptable to diverse rice plants. The proposed algorithm outperforms the classical structure-from-motion method in terms of texture preservation and computational efficiency.