基于卷积神经网络和重排序的农业遥感图像检索

卷积神经网络具有很强的分类能力,并在图像分类等应用中取得显著成效,但遥感图像检索应用中还较少利用该分类能力。为了提高农业遥感图像检索性能,该文提出一种利用卷积神经网络分类能力的遥感图像检索方法。首先利用微调的卷积神经网络模型提取查询图像的检索特征和估计查询图像的每个类别权重,然后利用根据CNN模型判断的检索图像类别和初始排序结果计算类别查准率,根据查询图像的类别权重和类别查准率计算加权类别查准率,最后根据加权类别查准率对图像类别进行排序,并根据排序结果对初始检索结果进行重排序,从而得到最终的检索结果。试验结果表明:该检索方法在PatternNet数据集中平均查准率达到97.56%,平均归一化调整后的检索秩达到0.0201;在UCM_LandUse数据集中平均查准率达到93.67%,平均归一化调整后的检索秩达到0.049 2,较之其他遥感图像检索方法下降0.2358,降幅超过82.7%;平均每张检索图像重排序时间大约是初始排序时间的1%。该文提出的重排序方法可以得到更好的遥感图像检索结果,提高了遥感图像检索性能,将有助于农业信息领域信息化和智能化。

Agricultural remote sensing image retrieval based on convolutional neural network and reranking

Convolutional neural network (CNN), a hierarchical neural network, can extract powerful feature representations and make accurate classification at the same time. CNN has already made remarkable achievements in various fields such as image classification and object recognition. The ability of feature extraction of CNN has been used to retrieve images in lots of works, however, the powerful classification ability of CNN is ignored by most researchers. To improve the agricultural image retrieval performance, this paper proposes a reranking method that uses the classification ability of CNN. Firstly, the fine-tuned cnn model is used to extract the retrieval features of the query image and estimate the weight of each category of the query image. Second, the retrieved images are sorted according to the image similarity of the CNN features between the query image and each retrieved image, and then the initial retrieval results are obtained. Third, the initial retrieval results are used to calculate the weighted class average precision (CAP) of each image class. Finally, the order of image classes is obtained through sorting the classes according to the weighted CAP, and the retrieved images are re-ranked by the order of image classes. The images in the same class are retained their order in the initial result. Hence, the final retrieval result is obtained. Experiments of two publicly available datasets of remote sensing, PatternNet and UCM_LandUse, are carried to verify the validation of the proposed method. The experimental results are concluded as follows: 1) The reranking method can improve the initial results and get more relevant images in a contrast experiment. 2) Per class mean average precision (mAP) values of three features (FC6 and FC7 of VGG16, pool5 of ResNet50) are evaluated on UCM_LandUse dataset, and the reranking retrieval results have increased by approximately 30% than the initial results. 3) To determine the optimal parameter values, an experiment of the different training data volume on PatternNet is conducted to evaluate the influence of different number of training images on the retrieval performance. It can be seen that the mAP and ANMRR(Average normalized modified retrieval rank) improves with the increases of the number of training image. For example, the mAP of ft_pool5_rerank feature increases from 75.89% to 97.56% as the number of the training image per class grows from 5 to 90. 4) The average resort retrieval time increases by no more than 1% over the initial retrieval time. 5) The mAP of the proposed method on UCMD is 93.67%, and the ANMRR is 0.049 2, which is 0.235 8 lower than that of the state-of-the-art methods.The proposed method can realize higher retrieval performance of agricultural remote sensing image retrieval, it will be helpful to improve the level of information and intellectualization in the agricultural information field.