基于多源图像融合的自然环境下番茄果实识别

蔬果采摘机器人面对的自然场景复杂多变,为准确识别和分割目标果实,实现高成功率采收,提出基于多源图像融合的识别方法。首先,针对在不同自然场景下单图像通道信息不充分问题,提出融合RGB图像、深度图像和红外图像的多源信息融合方法,实现了机器人能够适应自然环境中不同光线条件的番茄果实。其次,针对传统机器学习训练样本标注低效问题,提出聚类方法对样本进行辅助快速标注,完成模型训练;最终,建立扩展Mask R-CNN深度学习算法模型,进行采摘机器人在线果实识别。实验结果表明,扩展Mask R-CNN算法模型在测试集中的检测准确率为98.3%、交并比为0.916,可以满足番茄果实识别要求;在不同光线条件下,与Otsu阈值分割算法相比,扩展Mask R-CNN算法模型能够区分粘连果实,分割结果清晰完整,具有更强的抗干扰能力。

Tomato Fruit Recognition Based on Multi-source Fusion Image Segmentation Algorithm in Open Environment

The natural scenes faced by fruit and vegetable picking robots are complex and changeable. Accurate identification and segmentation of the target fruit are crucial for high success rate harvesting. The instance segmentation is an effective method to solve the problem. Howerver, existing instance segmentation algorithms have some drawbacks, such as the limited effect of edge segmentation accuracy for single-source images, the workload and time spent on image labeling. Therefore, a tomato fruit recognition algorithm based on multi-source fusion image and extended Mask R-CNN model was proposed. Firstly, aiming at the problem of insufficient information in different natural scenes with a single image channel, a multi-source information fusion method combining RGB images, depth images and infrared images was proposed, which enabled the robot to adapt to different lighting and fruits at different maturity stages. Secondly, aiming at the problem of inefficiency of traditional machine learning training sample standards, a clustering method was proposed to assist the rapid labeling of samples to complete the model training. Thirdly, an extended Mask R-CNN deep learning algorithm model was established for online fruit recognition by picking robots. The experimental results showed that the extended Mask R-CNN algorithm model achieved 98.3% detection accuracy and 0.916 detection IoU in the test set, which can well meet the requirements of tomato fruit recognition;under different lighting conditions, compared with the Otsu threshold segmentation algorithm, the extended Mask R-CNN algorithm model was able to distinguish the adherent fruits with clear and complete segmentation results and stronger anti-interference ability.