自然场景图像中的重叠蜜柚识别及试验

重叠蜜柚目标的准确分离和蜜柚果梗的定位是实现采摘自动化必须解决的两个关键问题。现有的苹果、柑橘等重叠果实分离方法不适用于重叠蜜柚，且无果梗定位功能。针对以上问题，本文提出了一种结合渐进式中心定位的重叠蜜柚分离方法和果梗定位方法。首先利用主成分分析方法提取蜜柚区域、滤除背景并对图像中的重叠蜜柚进行初步分离；接着，对重叠蜜柚区域采用渐进式中心定位方法得到各个蜜柚的中心；然后，利用区域边缘点到其相应的不同中心点的距离大小的变化规律实现重叠蜜柚的分离；最后，利用前述的中心点结合蜜柚的形状特征，定位出遮挡程度较小的蜜柚果梗。在50张自然场景下的图像上进行试验，结果表明在有阴影、小目标、遮挡和重叠等复杂环境下，该方法的平均识别率为94.02%。同时，对于果梗未被遮挡且离摄像头较近的蜜柚，也给出了准确的果梗区域。在利用蜜柚模型搭建的识别自动化试验平台上进行试验，结果表明采摘机器人能够有效识别并分离重叠蜜柚、定位果梗。本研究可为蜜柚采摘机器人准确识别重叠果实提供参考。

Recognition of the overlapped honey pomelo images in natural scene and experiment

Abstract: Two key challenges can be the accurate separation of the overlapping fruits and the positioning of the plant stem during the automatic picking of honey pomelo. However, the existing approaches to separate the overlapping apples and citrus are not suitable for the overlapping honey pomelo, particularly no positioning function of the stem so far. In this study, new image recognition was proposed to combine with the progressive center and stem positioning in natural scene images, in order to improve the recognition rate of honey pomelo. Firstly, the principal component analysis (PCA) was used to determine the principal components of the color pixel values in the target area and the distribution intervals of each component. The PCA was also utilized to reduce the data dimensionality. As such, the rotation matrix was obtained to convert the image from the RGB to the principal component space. The distribution intervals of the color principal components were used to evaluate and filter the pixels of the honey pomelo. A binarization was then performed to obtain a binary image. Secondly, the edge information of the color image after filtering the background was selected to preliminarily divide the binary image. If a pixel was an edge point on the color image, the corresponding point on the binary image was set for the background pixel. Before separating, a white area was determined to contain the multiple honey pomelos. In addition, the separation operation was performed on the area, only when there were multiple honey pomelos. Thirdly, a progressive center positioning was adopted to locate the center of each honey pomelo in the overlapping honey pomelo area. An operation was also conducted from the top, bottom, left, and right directions to obtain the center of each honey pomelo. Finally, the separation point of the overlapping area was determined to realize the recognition, where the edge points of the white area were traversed along the edge, in order to calculate the distance between each edge point and the center points of two adjacent honey pomelos. Since the stem of the honey pomelo was located near the top extension line of the long axis passing through the center point in the longitudinal section, the central point was used to locate the stem area with a smaller degree of obscuration and normal suspension. A total of 50 images in natural scenes were selected to verify the model. The test results showed that the average recognition rate of the new recognition was 94.02% in the natural scene. Furthermore, the stem areas were accurately located for the honey pomelos, whose stems were not blocked or closer to the camera. Consequently, the new recognition can be widely expected to transfer to the embedded development system and an automatic picking platform with the laboratory honey pomelo model for picking experiments. This finding can also provide a strong reference to accurately recognize the overlapping fruits for the picking robot of honey pomelo.