基于深度学习的番茄授粉机器人目标识别与检测

为满足植物工厂中番茄智能化授粉作业的需要，解决目前机器人在授粉过程中因花朵小、姿态朝向各异而导致的检测精度不高和授粉策略不完善的问题，该研究提出一种由目标检测、花期分类和姿态识别相结合的番茄花朵检测分类算法--TFDC-Net（Tomato flower detection and classification network）。在花朵检测阶段，基于YOLOv5s对其网络进行改进，添加了卷积块注意力模块（Convolutional Block Attention Module，CBAM）及采用了加权框融合（Weighted Boxes Fusion，WBF）的方法，提出一种改进的YOLOv5s网络。该网络在使用线下数据增强的基础上训练得到ACW_YOLOv5s模型，该模型的准确率为0.957，召回率为0.942，mAP0.5为0.968，mAP0.5～0.95为0.62，各项指标相较于原网络模型分别提高了0.028，0.004，0.012，0.066。经测试表明，ACW_YOLOv5s模型解决了远处的小目标及被遮挡的目标漏检，重叠目标误检的问题。同时，为解决花朵不同花期和花蕊不同朝向的授粉问题，研究采用EfficientNetV2分类网络分别对3种不同花期和5种不同姿态的花朵进行训练得到花期分类模型及姿态识别模型，模型准确率分别为94.5%和86.9%，通过对目标进行花期分类和姿态识别判断是否对其进行授粉。为进一步验证分类模型的性能，分别选取300张花期图片和200张姿态图片对模型进行测试，花期分类模型和姿态分类模型的测试总体准确率分别为97%，90.5%。将TFDC-Net算法应用于自主研发的授粉机器人中并进行试验验证，结果表明，该算法能够完成对番茄花朵的目标检测，花期分类和姿态识别。再通过坐标转换实现目标定位，并对目标进行授粉。该研究为授粉机器人的目标检测与定位提供了一定的技术依据。

Deep learning-based target recognition and detection for tomato pollination robots

Abstract: In order to meet the needs of intelligent pollination of tomatoes in plant factories, and to solve the problems of low detection accuracy and imperfect pollination strategies caused by small flowers and different posture orientations during pollination by robots, this study proposes a tomato flower detection and classification algorithm that combines target detection, flowering classification and posture recognition - the In this study, we propose a tomato flower detection and classification algorithm, TFDC-Net (Tomato flower detection and classification network). The algorithm is designed based on the characteristics of tomato flowers and is divided into two parts: the target detection part of tomato flowers and the flowering pose classification part of flowers. In the flower detection phase, this paper is based on the YOLOv5s network to improve the target detection accuracy. Two main improvements to the network structure are proposed: firstly, a CBAM(Convolutional Block Attention Module) module is added to enhance the effective features to suppress the invalid ones, and secondly, a WBF(Weighted Boxes Fusion) approach is adopted to make full use of the prediction information. The network was trained on the basis of using offline data augmentation to obtain the ACW_YOLOv5s model, which had an accuracy of 0.957, a recall of 0.942, an mAP0.5 of 0.968 and an mAP0.5-0.95 of 0.62, with each index improving by 0.028, 0.004, 0.012 and 0.066 respectively compared to the original network model. For to further verify the actual detection performance of the model for tomato flowers, this paper uses the model to compare with the original YOLOv5s model for the recognition of flowers under different complex situations. The tests show that the ACW_YOLOv5s model solves the problems of missed detection of small distant targets and obscured targets and false detection of overlapping targets that exist in the original YOLOv5s model. At the same time, to solve the pollination problem of flowers with different flowering stages and different stamen orientations, the EfficientNetV2 classification network was used to train three different flowering stages and five different postures of flowers to obtain the flowering stage classification model and posture recognition model respectively, and the accuracy of the models was 94.5% and 86.9% respectively. pollination. To further validate the performance of the classification model, 300 flowering images and 200 gesture images were selected to test the model. The overall accuracy of the flowering classification model and gesture classification model were 97% and 90.5% respectively. The TFDC-Net algorithm was obtained by integrating the ACW_YOLOv5s target detection model, the flowering classification model and the pose classification model, which enables the detection of tomato flowers and the classification of flowering pose to meet the vision requirements of pollination robots. The TFDC-Net algorithm is applied to the self-developed pollinator robot and the results show that the TFDC-Net algorithm can complete the target detection, flowering classification and pose recognition of flowers. The target is then localised through coordinate conversion, and the true 3D coordinates of the target in the robot arm''s coordinate system are obtained and fed back to the robot arm for pollination of the target in full bloom and with a front attitude. This study provides a technical basis for the target detection and localisation of pollination robots.