基于X-ray和RGB图像融合的实蝇侵染柑橘无损检测

实蝇侵染柑橘流入市场会造成巨大的经济损失,因此需要在商品化处理阶段对其全面筛除。针对柑橘在实蝇侵染早期没有明显外部特征,人工抽样检测效率低、筛除难的问题,探索了在生产线上同时搭载农业X光机与RGB相机进行无损检测的可行性,提出了基于X-ray(X光)和RGB图像的多模态数据融合方法,建立了CNN-LSTM检测模型,实现了实蝇侵染柑橘高精度无损检测。模拟了柑橘在生产线上滚动并被拍摄6幅X-ray和RGB序列图像的过程,构建了实蝇侵染柑橘的多源数据集,融合了不同模态的实蝇侵染特征信息,提升了实蝇侵染柑橘检测模型的检测能力,并对比了ResNet18-LSTM、GoogleNet-LSTM、SqueezeNet-LSTM、MobileNetV2-LSTM轻量化检测模型,验证了多模态数据融合方法的有效性。研究结果表明,提出的多模态数据融合实蝇侵染柑橘方法比单模态检测方法检测性能更加优异,其中ResNet18-LSTM检测准确率最高,多模态的图像融合和特征融合方法检测准确率分别达到97.3%和95.7%,单模态X-ray和RGB检测方法准确率分别为93.2%和89.3%。本研究可为实蝇侵染柑橘在线...

Nondestructive Detection of Citrus Infested by Bactrocera dorsalis Based on X-ray and RGB Image Data Fusion

Citrus fruit infested by Bactrocera dorsalis can cause consumer panic and huge economic losses, which makes it important to sort it out during processing. Since there are no obvious characteristics on the fruit surface and manual sorting usually features low efficiency, new techniques for automated sorting are needed. The feasibility of combining an agricultural X-ray machine and an RGB camera on the processing line was explored for non-destructive detection. A multi-modal data fusion method using X-ray and RGB images was firstly proposed, and a CNN-LSTM detection model was then developed which can detect the fruit infested by Bactrocera dorsalis with high precision. The process of the fruit rolling on the processing line was simulated and six X-ray and RGB sequential images were captured respectively, which formed the dataset. The effectiveness of multi-modal data fusion was verified by integrating it into four lightweight detection models, including ResNet18-LSTM, GoogleNet-LSTM, SqueezeNet-LSTM and MobileNetV2-LSTM. Results showed that for each network, the performance using multi modal data fusion outperformed that using unimodal data. ResNet18-LSTM obtained the highest detection accuracy, reaching 97.3% by using multi-modal image fusion and 95.7% by using feature fusion, respectively, and the accuracy based on single-modal X-ray and RGB data was 93.2% and 89.3%, respectively. These results demonstrated the potential to develop an online non-destructive detection system for citrus fruit infested by Bactrocera dorsalis.