基于深度学习的温室大棚实例识别及模型迁移

温室大棚实例提取在蔬菜种植面积测算和产量估计等方面具有重要意义。该研究以高效准确地识别大尺度范围内温室大棚实例为目标，提出了一种基于卷积神经网络和形态学后处理的“区域-边界”实例提取方法，以及单纯迁移模式、尺度适应模式、模型微调模式3种不同的迁移模式。试验结果表明，利用UNet网络构建“区域-边界”多分类模型识别温室大棚实例效果最优（实例召回率达到91.05%）。形态学后处理操作能够进一步优化温室大棚实例提取结果（单元交并比相比于操作前提高10.53个百分点）。探讨了3种模型迁移模式应用在不同场景时的表现，迁移效果由高到低依次为模型微调模式（实例召回率为87.93%）、尺度适应模式（实例召回率为41.72%）、单纯迁移模式（实例召回率为24.15%）。基于“区域-边界”实例提取方法并根据预测区域和训练区域的场景差异选择不同的迁移模式可以快速精准地识别大尺度范围内温室大棚实例，为农业设施的智能化建设提供信息支撑。

Recognizing greenhouse instance and model transfer using deep learning

The timely and accurate extraction of greenhouse instance (GI) is of significant practical importance to estimate the vegetable cultivation areas and yield prediction. Deep learnings (driven by knowledge learned from large-scale sample data) can be expected to adaptively explore the features of image data, compared with the traditional image analysis, such as unsupervised, supervised, and object-oriented classification. The end-to-end accurate extraction of GI information can also ensure the model generalization for the less manual intervention. However, there are still two challenges to identify the GI using deep learning. One is that the multiple GI can be mistakenly assumed as a continuous distribution in the dense areas of GI, leading to the segmentation errors. Another is that the degradation of performance could occur, when transferring the GI model to the large-scale spatial context. In this study, the region-boundary instance extraction was proposed using convolutional neural networks (CNNs) and morphological post-processing. At first, a region-boundary multi-class model was constructed to generate the boundary auxiliary labels of greenhouse. The network loss function was modified to enhance the boundary information recognition, and then to facilitate the removal of greenhouse boundaries from the recognition, thereby achieving the extraction of GI. Subsequently, the high-precision GI data was obtained using morphological operations, such as the instance object dilation and the minimum bounding rectangles. The high-resolution three-band remote sensing data was collected from Shouguang, Shandong Province, in order to train the base model. Three transfer modes (namely, pure transfer, scale-adaptive, and model fine-tuning mode) were then explored in five transfer research areas, including Xinjiang, Liaoning, Yunnan, Hubei, and Zhejiang Province. The accuracy of GI extraction was evaluated using common semantic segmentation performance metrics, unit intersection over union (UIoU), and instance recall rate (IRR). The research results indicate that the UNet was better suited to construct as the "Region-Boundary" multi-class model, compared with the semantic segmentation networks, such as PSPNet, DeeplabV3+, and HRNet. The higher semantic accuracy was achieved in the UNet with the UIoU and IRR of 2.43 and 2.91 percentage points higher, respectively, compared with the overall suboptimal HRNet. Furthermore, two morphological post-processing operations (instance dilation and the minimum bounding rectangle recognition) were simultaneously introduced to increase the UIoU and IRR by 10.53 and 1.44 percentage points, respectively. The scale adaptation mode was then adopted to adjust the input image resolution. The UIoU and IRR were improved from 3 to 22 percentage points, and from 2 to 30 percentage points, respectively, in all test datasets of migration areas, compared with the simple migration mode. The fine-tuning of the base model was utilized to adjust the input image resolution in the model fine-tuning mode. The UIoU and IRR were improved ranging from 37 to 50 percentage points, and from 45 to 76 percentage points. The higher accuracy was achieved in the PGI recognition, with the UIoU and IRR of 13.64 and 14.18 percentage points higher than the conventional approaches, respectively. Simultaneously, the model transfer was applied to select the different migration modes, according to the scene differences between the predicted and training regions. The automated mapping of PGI can be expected to efficiently and accurately extract the PGI information over the large-scale areas. The finding can provide the information support to the intelligent construction of agricultural facilities.