基于注意力机制及多尺度特征融合的番茄叶片缺素图像分类方法

针对番茄早期缺素性状不明显及各生长期特征差异较大所导致的特征区域尺寸不一致、难提取、难辩别等问题，提出了一种基于注意力机制及多尺度特征融合卷积神经网络的番茄叶片缺素图像分类方法（Multi-Scale Feature Fusion Convolutional Neural Networks Based On Atte ntion Mechanism，MSFF-AM-CNNs）。首先根据番茄叶片缺素特点提出了多尺度特征融合结构（Multi-Scale Feature Fusion Module，MSFF Module）；其次在DenseNet基础上，结合浅层网络主要提取纹理、细节特征，深层网络主要提取轮廓、形状特征的特点分别提出具有针对性的特征提取方法，通过不同形式引入注意力机制及多尺度特征融合结构，使全局多尺度信息融合多个特征通道、选择性地强调信息特征并达到对特征精准定位的功能；同时引入Focal Loss函数以减少易分类样本的权重。试验结果表明，MSFF-AM-CNNs的平均召回率、平均F1得分、平均准确率较原模型DenseNet-121均大幅提升，其中缺氮和缺钾叶片的准确率分别提高了8.06和6.14个百分点，召回率分别提高了6.31和5.00个百分点，F1得分分别提高了7.25和5.55个百分点，平均识别准确率可达95.92%，具有较高的识别准确率及广泛的适用性，能够满足番茄叶片缺素图像的高精度分类需求，可为植物叶片缺素识别提供参考。

Image classification method for tomato leaf deficient nutrient elements based on attention mechanism and multi-scale feature fusion

This study aims to realize the accurate identification of deficient nutrients elements in tomato leaves. An experiment was conducted on the lack of nutrients in the climate chamber in the laboratory of Big Data Intelligence Department of the Beijing Academy of Agriculture and Forestry Sciences, China. An artificial climate chamber was also selected to regulate the growth environment factors of tomato plants for the specific lack of nutrients. Three types of nutrient deficiency groups were set, namely nitrogen deficiency, phosphorus deficiency, and potassium deficiency, as well as a normal control group. The experiment was started with the appearance of nutrient-deficient traits in seedlings, and the images of nutrient-deficient leaves were then collected according to the growth stages. The experimental results show that there were diversity and differences in the traits of tomato nutrient deficiency. Specifically, there were relatively small changes of leaves in the early stage of tomato nutrient deficiency. Furthermore, it was difficult to capture the details and textures, due to the smaller area of traits. For example, the manifestation of phosphorus deficiency was that the leaves gradually turn purple along the veins. The trait details were hardly identified in the early stage of phosphorus deficiency, due mainly to the mostly small vein structure. Particularly, tomato leaves under different conditions of nutrient deficiency presented similar color and texture characteristics at a certain stage. For example, the leaves were both slightly yellow in the early stage of nitrogen deficiency and the early stage of potassium deficiency. The only slight difference was the characteristic display of morphology in the size of characteristic areas. There were obvious differences in color and texture at different stages under the same nutrient deficiency. The images were collected from the climate chamber to serve as the experimental data. An attempt was made on the inconsistency of feature area size, and the difficulty of feature extraction, resulting from the different types of nutrient deficiency, the insignificant early traits of nutrient deficiency, and the large differences in the characteristics of each growth period. Therefore, an image classification was proposed for the nutrient deficiency of tomato leaves using an attention mechanism and multi-scale feature fusion convolutional neural network (MSFF & AM-CNNs). First of all, a multi-scale feature fusion (MSFF) module was set for nutrient deficiency traits, due to the low efficiency of a fixed-scale convolution kernel for different sizes. The MSFF input image was carried out with multi-channel feature stitching after the MSFF convolution kernel of multiple scales, where the shallow image was multiplied while expanding the number of channels. As such, the fusion of scale features was adopted in this structure. Secondly, an MSFF&AM module was used to improve the large-scale convolutional layer for the extraction of shallow features using the attention mechanism (CBAM). A multi-scale fusion of Bottleneck was also utilized to improve the Dense Block for the extraction of deep features. Deep-MSFF Block aimed to combine the attention mechanism and the MSFF module, where the multiple feature channels were selectively emphasized the global multi-scale information feature function. The recalibration of features in nitrogen deficiency was improved on the tomato leaves the classification accuracy. Finally, a Focal Loss function was introduced as the loss function to reduce the weight of easy-to-differentiate samples. Correspondingly, the image recognition model of tomato elements lacking was widely expected to focus on difficult-to-classify samples during training, particularly for the overall performance of the model. The experiments demonstrated that the MSFF & AM-CNNs can meet the high-precision classification requirements of nutrient-deficient images in tomato leaves, particularly with high recognition accuracy and wide applicability (an average recognition accuracy rate of 95.92%). The model can also be expected for the identification of plant leaf nutrient deficiency.