面向苹果叶部病害识别的细粒度蒸馏模型

为了提高轻型卷积神经网络（convolutional neural networks，CNN）在苹果叶部病害识别中的精度，使其更加适于布署到智慧农业移动终端，该研究设计了一种细粒度知识蒸馏（fine-grained knowledge distillation，FGKD）模型。首先，利用上下文信息与空间-语义关系分别设计了上下文空间注意力（spatial attention，SA）与细粒度特征提取（fine-grained feature extraction，FGFE）模块，且将它们嵌入到Resnet50与设计的轻型CNN，分别作为教师与学生网络；然后，构造SA与FGFE知识蒸馏损失函数，以将教师网络中的特征提取与细粒度知识表示能力迁移到学生网络之中，以增强其对苹果叶部病害图像的局部特征提取能力与高层语义表达能力，使轻型学生网络在参数量很小的条件下，其性能接近复杂的教师网络。基于标准苹果叶部病害数据集的对比试验结果表明，经知识蒸馏之后的学生网络精度为98.60%，模型参数量仅0.75 MB，平均推理时间为25.51 ms，能够有效地满足实际智慧农业移动端对模型的需求，快速准确地实现苹果叶部病害自动识别。

Identifying apple leaf disease using a fine-grained distillation model

Apple trees are susceptible to various diseases that caused by weather, environment and microorganisms. The leaves of plants are the most common parts of the disease. The small area and similar symptoms of diseases have also posed great challenges on the manual observation and experience judgment in recent years. The disease type cannot be diagnosed in time, resulting in the huge losses in apple production. Deep learning can automatically extract features in crop diseases, but it also suffers from an excessive number of parameters and high computational effort. Furthermore, various lightweight architectures have been constructed to provide the strong technical support for the deployment of crop disease identifications, such as less network parameters, less computation, simple models, and low practicability of deep learning models. However, the direct application or improvement of the existing light convolutional neural network (CNN) can fail to further optimize the fine-grained problem in "small variance between classes and large variance within classes" of apple leaf diseases. Multiple CNN frameworks or attention modules can be utilized to consider the coarse-grained global and fine-grained local features of apple leaf diseases. It is necessary for the small number of parameters to meet the requirements of smart agriculture for mobile deployment. In this study, a fine-grained knowledge distillation (FGKD) model was proposed to improve the CNN accuracy in the disease identification of apple leaf suitable for the deployment to smart agricultural mobile terminals. Firstly, contextual information and spatial-semantic relations were used to design the spatial attention (SA) and fine-grained feature extraction (FGFE) modules respectively, and they were embedded into Resnet50 and the designed light CNN as teacher and student networks. Secondly, the SA and FGFE knowledge distillation loss functions were constructed to transfer the feature extraction and fine-grained knowledge representation of the teacher to the student network, in order to enhance the local feature extraction and high-level semantic expression of apple leaf disease images. Finally, the performance of the light student network was close to that of the complex teacher network under the condition of a small number of parameters. The comparative test was carried out on the standard apple leaf disease dataset. The results show that the accuracy of the student network was 98.60% after knowledge distillation, while the number of model parameters was only 0.75 MB, and the average inference time was 25.51 ms. The automatic identification of apple leaf diseases was be rapidly and accurately realized to fully meet the needs of the model of the actual smart agriculture mobile terminals. The SA module and SA distillation function were designed to combine the contextual information and spatial attention, in order to effectively improve the extraction of local information about the disease. The spatial-semantic relationship aggregation of fine-grained features was used to enhance the extraction and expression of high-level semantic information about the disease.