基于语义分割的矮化密植枣树修剪枝识别与骨架提取

为了实现休眠期枣树自动选择性剪枝作业,针对复杂树形结构修剪枝难以识别的问题,研究了基于语义分割网络实现自然场景中枣树修剪枝识别与骨架提取。通过RGB-D相机搭建的视觉系统获取不同天气情况下枣树的点云信息,根据距离阈值去除复杂的枣园背景,并构建枣树前景数据集。利用DeepLabV3+和PSPNet 2种深度学习模型分割枣树枝干同时获取其修剪枝的掩膜,并进行结果对比。对修剪枝掩膜进行二值化,依据二值图像的面积去除噪声,对去噪后的连通域标记,并提取修剪枝骨架,最终确定修剪枝数量,建立修剪枝数量真实值与预测值之间的线性回归模型。结果表明：基于ResNet-50特征提取网络的DeepLabV3+模型识别结果最好,平均像素准确率（mPA）、平均交并比（mIoU）分别为89%和81.85%,其中枣树主干、修剪枝2个类别的像素准确率（PA）和交并比（IoU）分别为90.36%、80.98%和80.34%、66.69%;在3种典型天气（晴天、阴天、夜间）情况下,晴天枣树枝干的mPA（91.97%）略高于阴天（91.81%）和夜间（90.98%）,同时,预测的修剪枝与真实值的R2（0.8699）也高于阴天（0.8373）和夜间（0.8120）,并得到最小的RMSE为1.1618。

Method for Detection and Skeleton of Pruning Branch of Jujube TreeBased on Semantic Segmentation for Dormant Pruning

Dormant pruning is a labor-intensive and time-consuming operation. It is an important part for the refined management of jujube orchard, which can control the tree structures by removing the over-long branches, thus decreasing the limbs density. Automated pruning using a robotic platform could be a better solution. To realize automatic selective pruning for dormant jujube tree, the segmentation of branch and trunk of tree was difficult in complex jujube orchard background. A method based on semantic segmentation network for branch recognition of jujube trees was studied in field. The visual system built by RGB-D camera was used to acquire the point cloud information of jujube trees under different weather conditions, and the background was removed by using the distance threshold for construction of foreground jujube tree datasets. Two kinds of semantic segmentation models, DeepLabV3+ and PSPNet, were utilized to segment branch and trunk of jujube tree and obtain the pruning branch mask, meanwhile the results of segmentation were compared. The mask of pruning branch was binarized, and the noise was removed based on the area of the connected domain of the binary image. The connected domain was labeled after denoising, and the branch skeleton was extracted. Finally, the number of pruning branch was determined, and the linear regression model for the real value and predicted value of the pruning number was established. The results showed that the DeepLabV3+ model based on ResNet-50 (feature extraction network) achieved the best segmentation results, and its average pixel classification accuracy and average intersection-over-union were 89% and 81.85%, respectively. The PA and IoU for trunk and pruning branch were 90.36%, 80.98% and 80.34%, 66.69%, respectively. The mean pixel accuracy for branch and trunk of jujube tree in sunny was 91.97%, which was slightly higher than that in cloudy (91.81%) and night (90.98%) under three typical weather conditions. Meanwhile, the R2 was 0.8699 between predicted values and real value in sunny, which was higher than that of cloudy day (0.8373) and night (0.8120), and the minimum RMSE (1.1618) was obtained.