A 3D grape bunch reconstruction pipeline based on constraint-based optimisation and restricted reconstruction grammar

Phenotypic traits of grapevines are known to be closely related to grapevine yield, wine flavour and sensitivity to disease. Traditional phenotyping methods based on manual measurements face the bottleneck of intensely repetitive and time consuming measurement. As viticulturists turning their focus to the 3D automatic phenotyping domain, existing work on grape bunch phenotyping indicates deficiencies in incomplete reconstruction information, poor element coincidence with the ground truth and poor performance under field conditions. To this end, the proposed work introduces a novel reconstruction pipeline by dividing it into sub-problems of visible berry-related reconstruction and invisible element prediction. By taking a 2D image of the target bunch as the only sensor input, visible berries are detected using image processing algorithms. With the detected berry information, their morphological positions are predicted, from which internodes that are associated with the detected berries are derived. Parameters of derived internodes are then estimated employing constraint-based optimisation theory, from which the visible-berry-related reconstruction is able to be achieved. Invisible element prediction is then conducted by filling elements according to a Restricted Reconstruction Grammar (RRG). A fully reconstructed bunch model is finally presented. Compared with existing work, the proposed grape bunch reconstruction pipeline achieved an improvement in quantity estimation of rachis internodes, tertiary internodes and pedicels, whose percentage errors were indicated as 21.2, 42.2 and 31.2% respectively. A better performance was also revealed in length estimations of secondary internodes and pedicels with percentage errors of 3.5 and 0.5%. This may largely facilitate related studies on disease control of grape bunches since internode numbers and lengths are closely related to bunch compactness which is the indicator of disease sensitivity. Especially, the proposed reconstruction pipeline shows an promising improvement in element coincidence with F1 scores of 0.90, 0.77, 0.45, 0.43 for respective element types. Knowing that element coincidence may influence the inner space utilisation of a bunch, the proposed work provides a better option for 3D grape bunch phenotyping.

Automated summary

This article proposes a new grape bunch reconstruction pipeline which detects visible berries and predicts their morphological positions using a 2D image, achieving reconstruction. It also fills invisible elements using a Restricted Reconstruction Grammar, resulting in a fully reconstructed bunch model. Compared to existing work, this pipeline shows significant improvement in quantity and length estimation, as well as element coincidence.