Green citrus detection using ‘eigenfruit’, color and circular Gabor texture features under natural outdoor conditions

A machine vision algorithm was developed to detect and count immature green citrus fruits in natural canopies using color images. A total of 96 images were acquired in October 2010 from an experimental citrus grove in the University of Florida, Gainesville, Florida. Thirty-two of the total 96 images were selected randomly and used for training the algorithm, and 64 images were used for validation. Color, circular Gabor texture analysis and a novel ‘eigenfruit’ approach (inspired by the ‘eigenface’ face detection and recognition method) were used for green citrus detection. A shifting sub-window at three different scales was used to scan the entire image for finding the green fruits. Each sub-window was classified three times by eigenfruit approach using intensity component, eigenfruit approach using saturation component, and circular Gabor texture. Majority voting was performed to determine the results of the sub-window classifiers. Blob analysis was performed to merge multiple detections for the same fruit. For the validation set, 75.3% of the actual fruits were successfully detected using the proposed algorithm.     

Detection and counting of immature green citrus fruit based on the Local Binary Patterns (LBP) feature using illumination-normalized images

Early detection and counting of immature green citrus fruit using computer vision can help growers produce a predictive yield map which could be used to adjust management practices during the fruit maturing stages. However, such detecting and counting is difficult because of varying illumination, random occlusion and color similarity with leaves. An immature fruit detection algorithm was developed with the aim of identifying and counting fruit in a citrus grove under varying illumination environments and random occlusions using images acquired by a regular red–green–blue (RGB) color camera. Acquired citrus images included front-lighting and back-lighting illumination conditions. The Retinex image enhancement algorithm and the two-dimensional discrete wavelet transform were used for image illumination normalization. Color-based K-means clustering and circular hough transform (CHT) were applied in order to detect potential fruit regions. A Local Binary Patterns feature-based Adaptive Boosting (AdaBoost) classifier was built for removing false positives. A sub-window was used to scan the difference image between the illumination-normalized image and the resulting image from CHT detection in order to detect small areas and partially occluded fruit. An overall accuracy of 85.6% was achieved for the validation set which showed promising potential for the proposed method.     

Green citrus detection using fast Fourier transform (FFT) leakage

Detection of immature green citrus fruit is important during the early life cycle of citrus fruit. It allows growers to manage citrus groves more efficiently and maximize yields by identifying expected fruit yields well in advance before harvesting. It also helps the growers prepare harvesting equipment and pickers for the harvesting operation. A novel technique was developed for detecting immature green citrus fruit from an outdoor color image and counting number of fruits. This technique is unique in that it is the first known attempt towards exploring it on green citrus fruits. A set of 71 images containing immature green citrus fruit was acquired in an experimental citrus grove at the University of Florida, Gainesville, Florida, USA. An algorithm was developed using a set of 11 training images by calculating the fast Fourier transform leakage values for fruit and leaves. A threshold value was obtained by comparing the percent leakage of fruit and other objects. The algorithm was tested on a set of 60 validation images. The correct total fruit count for a validation set came out to be 120, whereas the actual number of fruit was 146. The overall correct detection rate was 82.2 %. The proposed algorithm can be further improved to help growers manage their grove more efficiently.     

Feature extraction of hyperspectral images for detecting immature green citrus fruit

At an early immature growth stage of citrus, a hyperspectral camera of 369–1042 nm was employed to acquire 30 hyperspectral images in order to detect immature green fruit within citrus trees under natural illumination conditions. First, successive projections algorithm (SPA) were implemented to select 677, 804, 563, 962, and 405 nm wavebands and to construct multispectral images from the original hyperspectral images for further processing. Then, histogram threshold segmentation using NDVI of 804 and 677 nm was implemented to remove image backgrounds. Three slope parameters, calculated from the pairs 405 and 563 nm, 563 and 677 nm, and 804 and 962 nm were used to construct a classifier to identify the potential citrus fruit. Then, a marker-controlled watershed segmentation based on wavelet transform was applied to obtain potential fruit areas. Finally, a green fruit detection model was constructed according to Grey Level Co-occurrence Matrix (GLCM) texture features of the independent areas. Three supervised classifiers, logistic regression, random forest and support vector machine (SVM) were developed using texture features. The detection accuracies were 79%, 75%, and 86% for the logistic regression, random forest, and SVM models, respectively. The developed algorithm showed a great potential for identifying immature green citrus for an early yield estimation.     

Immature green citrus fruit detection and counting based on fast normalized cross correlation (FNCC) using natural outdoor colour images

A fast normalized cross correlation (FNCC) based machine vision algorithm was proposed in this study to develop a method for detecting and counting immature green citrus fruit using outdoor colour images toward the development of an early yield mapping system. As a template matching method, FNCC was used to detect potential fruit areas in the image, which was the very basis for subsequent false positive removal. Multiple features, including colour, shape and texture features, were combined in this algorithm to remove false positives. Circular Hough transform (CHT) was used to detect circles from images after background removal based on colour components. After building disks centred in centroids resulted from both FNCC and CHT, the detection results were merged based on the size and Euclidian distance of the intersection areas of the disks from these two methods. Finally, the number of fruit was determined after false positive removal using texture features. For a validation dataset of 59 images, 84.4 % of the fruits were successfully detected, which indicated the potential of the proposed method toward the development of an early yield mapping system.     

桃新品系‘保佳俊’果实发育后期可溶性糖含量及蔗糖代谢相关酶活性变化

‘保佳俊’是河北农业大学选育的桃新品系,其果实可溶性糖含量显著高于‘大久保’。为探讨不同品种(系)桃果实间糖分积累差异及其与蔗糖代谢酶之间的关系,本试验以‘保佳俊‘和‘大久保’为试材,测定了果实发育后期蔗糖、葡萄糖、果糖和山梨醇含量及蔗糖代谢相关酶活性的变化,并对桃果实中可溶性糖分积累与酶活性的相关性进行了分析。结果表明,在桃果实成熟期,可溶性糖组分中蔗糖含量最高,其次是葡萄糖和果糖,山梨醇含量最低;‘保佳俊’果实的4种可溶性糖含量在成熟期均极显著高于‘大久保’。在果实成熟期,‘保佳俊’的SS、SPS、AI、NI酶活性均显著或极显著高于‘大久保’。相关性分析表明,‘保佳俊’果实蔗糖含量与SS酶活性呈显著正相关,与AI、NI酶活性均呈极显著负相关。‘大久保’蔗糖含量与SS酶活性呈极显著正相关。2个品种间酶活性差异可能是‘保佳俊’蔗糖含量显著高于‘大久保’的主要原因。     

不同葡萄糖/果糖类型桃在果实发育期间果实和叶片中可溶性糖含量变化及其相关关系

 【目的】研究不同葡萄糖/果糖（glucose/fructose,G/F）类型桃果实内G/F差异的部位和时期。【方法】以不同G/F类型的6个桃品种（G/F≈1品种：‘冈山白’、‘山一白桃’和‘燕红’;高G/F品种：‘张黄7号’、‘龙246’和‘临白7号’）为试材,采用高效液相色谱法测定果实发育期果实和叶片中可溶性糖含量,并在盛花后74 d或101 d测定了‘冈山白’、‘山一白桃’、‘张黄 7号’和‘龙 246’新梢韧皮部中可溶性糖的含量。【结果】两类不同G/F桃果实中均以蔗糖作为主要碳水化合物积累形式,花后43～85 d蔗糖含量很低,随后持续快速积累直至果实成熟;花后43～85 d山梨醇有升高趋势,在果实成熟前40 d左右迅速降低;葡萄糖和果糖含量在果实发育早期较高,之后逐渐降低;但两类不同G/F桃在整个果实发育过程中G/F值与果实成熟时相似。叶片中贮藏的可溶性糖主要是蔗糖和山梨醇,在果实整个发育期间,G/F≈1品种叶片中G/F约为1～3,而高G/F品种叶片中G/F约为2～7。G/F≈1品种‘冈山白’和‘山一白桃’与高G/F品种‘张黄7号’和‘龙246’韧皮部中山梨醇占总可溶性糖47%～58％,显著高于蔗糖、葡萄糖和果糖的含量,G/F为0.8～0.91,且两类不同G/F桃品种间G/F不存在显著差异。【结论】光合产物在韧皮部的运输对桃果实的G/F没有显著影响,果实中G/F的差异主要由于果实内糖代谢差异所导致。     

Deep learning for real-time fruit detection and orchard fruit load estimation: benchmarking of ‘MangoYOLO’

The performance of six existing deep learning architectures were compared for the task of detection of mango fruit in images of tree canopies. Images of trees (n?=?1 515) from across five orchards were acquired at night using a 5 Mega-pixel RGB digital camera and 720 W of LED flood lighting in a rig mounted on a farm utility vehicle operating at 6 km/h. The two stage deep learning architectures of Faster R-CNN(VGG) and Faster R-CNN(ZF), and the single stage techniques YOLOv3, YOLOv2, YOLOv2(tiny) and SSD were trained both with original resolution and 512?×?512 pixel versions of 1 300 training tiles, while YOLOv3 was run only with 512?×?512 pixel images, giving a total of eleven models. A new architecture was also developed, based on features of YOLOv3 and YOLOv2(tiny), on the design criteria of accuracy and speed for the current application. This architecture, termed ‘MangoYOLO’, was trained using: (i) the 1 300 tile training set, (ii) the COCO dataset before training on the mango training set, and (iii) a daytime image training set of a previous publication, to create the MangoYOLO models ‘s’, ‘pt’ and ‘bu’, respectively. Average Precision plateaued with use of around 400 training tiles. MangoYOLO(pt) achieved a F1 score of 0.968 and Average Precision of 0.983 on a test set independent of the training set, outperforming other algorithms, with a detection speed of 8 ms per 512?×?512 pixel image tile while using just 833 Mb GPU memory per image (on a NVIDIA GeForce GTX 1070 Ti GPU) used for in-field application. The MangoYOLO model also outperformed other models in processing of full images, requiring just 70 ms per image (2 048?×?2 048 pixels) (i.e., capable of processing?~?14 fps) with use of 4 417 Mb of GPU memory. The model was robust in use with images of other orchards, cultivars and lighting conditions. MangoYOLO(bu) achieved a F1 score of 0.89 on a day-time mango image dataset. With use of a correction factor estimated from the ratio of human count of fruit in images of the two sides of sample trees per orchard and a hand harvest count of all fruit on those trees, MangoYOLO(pt) achieved orchard fruit load estimates of between 4.6 and 15.2% of packhouse fruit counts for the five orchards considered. The labelled images (1 300 training, 130 validation and 300 test) of this study are available for comparative studies.

     

An adaptive image segmentation algorithm for X-ray quarantine inspection of selected fruits

Although X-ray scanners are commonly used in airports or customs for security inspection, practical application of X-ray imaging in quarantine inspection to prevent propagation of alien insect pests in imported fruits is still unavailable. The first step to identify insect infestation in fruit by X-ray imaging technique is image acquisition. This is followed by the image segmentation procedure, which can locate sites of infestation. Since the grey level of X-ray images depends on the density and thickness of the test samples, the relative contrast of infestation site to the intact region inside a typical fruit varies with its position. To accurately determine whether a fruit has signs of insect infestation, we have developed an adaptive image segmentation algorithm based on the local pixels intensities and unsupervised thresholding algorithm. This paper presents the detailed image processing procedure including the grid formation, local thresholding, threshold value interpolation, background removal, and morphological filtering for the determination of infestation sites of a fruit in X-ray image. The real-time image processing procedure was tested with X-ray images of several types of fruit such as citrus, peach, guava, etc. Additional tests and analyses were also performed using the developed algorithm on the X-ray images obtained with different image acquisition parameters.     

自然光照条件下苹果识别方法对比研究

针对自然光照条件下果园苹果识别效果不佳的问题,从苹果的颜色分割和形状提取2方面进行对比研究,提出一种自然光照条件下的苹果识别方法。利用错检率、漏检率和处理速度3个量化指标综合对比分析颜色阈值、SVM和BPNN 3种苹果颜色分割方法的处理效果。比较6种边缘检测算法对苹果区域图像的边缘检测效果,并使用Hough圆检测算法对苹果形状进行提取,以获得苹果的圆心和半径。试验结果表明:由BPNN的苹果颜色分割方法以及结合Log和Hough的苹果形状提取方法所构建的果实识别算法具有较高的鲁棒性和准确性,能有效克服果实遮挡、重叠和颜色变异等问题,果实平均识别率可达91.6%。     

桃果实品质动态变化及其对气象因子的响应

为提高桃品质并确定最佳采收期,以‘久保’为供试材料,分析果实的单果重、果径、硬度、可溶性固形物含量和可滴定酸含量的变化规律,并利用多元回归法建立果实品质与关键气象因子的综合模型。结果表明：桃果实单果重和果径生长发育动态呈现“慢—快—慢”阶段性特征,且均符合Logister曲线,单果重增长极大值出现在花后91 d,纵径增长极大值出现在花后66 d,横径增长极大值出现时间比纵径推迟5 d;随着果实的成熟,硬度逐渐下降,可溶性固形物含量呈上升趋势,上升幅度逐渐加快,可滴定酸含量先小范围波动后迅速下降。结合回归分析结果可判定,果实品质主要受≥18℃积温、≥24℃积温、日均气温和累计日照时数的共同影响。在≥18℃积温≥1170.21℃、≥24℃积温≥769.23℃、日均气温≥27.00℃、累计日照时数≥318.89 h条件下,果实品质达到最佳成熟度。     

Citrus fruits maturity detection in natural environments based on convolutional neural networks and visual saliency map

Citrus fruits do not ripen at the same time in natural environments and exhibit different maturity stages on trees, hence it is necessary to realize selective harvesting of citrus picking robots. The visual attention mechanism reveals a physiological phenomenon that human eyes usually focus on a region that is salient from its surround. The degree to which a region contrasts with its surround is called visual saliency. This study proposes a novel citrus fruit maturity method combining visual saliency and convolutional neural networks to identify three maturity levels of citrus fruits. The proposed method is divided into two stages: the detection of citrus fruits on trees and the detection of fruit maturity. In stage one, the object detection network YOLOv5 was used to identify the citrus fruits in the image. In stage two, a visual saliency detection algorithm was improved and generated saliency maps of the fruits; The information of RGB images and the saliency maps were combined to determine the fruit maturity class using 4-channel ResNet34 network. The comparison experiments were conducted around the proposed method and the common RGB-based machine learning and deep learning methods. The experimental results show that the proposed method yields an accuracy of 95.07%, which is higher than the best RGB-based CNN model, VGG16, and the best machine learning model, KNN, about 3.14% and 18.24%, respectively. The results prove the validity of the proposed fruit maturity detection method and that this work can provide technical support for intelligent visual detection of selective harvesting robots.

     

桃若干重要性状的KASP分子标记开发与应用

【目的】 开发一系列高通量、低成本的桃重要性状竞争性等位基因特异PCR（Kompetitive Allele Specific PCR,KASP）标记,包括果皮有毛/无毛、果形扁平/圆形、果肉硬质/非硬质、DBF（Dominant Blood Flesh）红肉/非红肉、抗/感蚜等5对性状,加速桃优良品种培育,缩短桃育种年限。【方法】 本研究在控制这些性状的候选基因及位点附近300 kb内,利用已有桃种质资源的基因组序列比对,开发KASP标记,对已知表型的桃种质材料进行基因分型验证,最终获得与目标性状紧密连锁的KASP标记。【结果】 利用开发的5个性状的KASP分子标记对桃杂交分离群体和自然群体进行基因型检测,结果表明,标记鉴定结果与已知表型完全一致,准确率为100%。其中,杂交群体中果皮有毛/无毛性状分离比例为30﹕30,果形扁平/圆形分离比例为31﹕29,果实硬质/非硬质分离比例为27﹕26,抗蚜/感蚜分离比例为49﹕46,均符合孟德尔遗传1﹕1的分离定律。【结论】 开发的KASP标记可高效检测桃果实外观、抗性、肉质等重要性状相关基因的等位变异,在基因型鉴定、亲本选配和杂种后代的分子标记辅助选择中有很好的应用前景。     

晚熟桃园梨小食心虫越冬茧的空间分布研究

[目的]了解晚熟桃园梨小食心虫越虫茧的空间分布情况。[方法]以中华寿桃、北京晚蜜和黄金冬桃为试材,研究了晚熟桃园梨小食心虫越冬茧的空间分布情况。[结果]越冬茧主要分布在桃树的地上部,土壤及地表枯枝中较少,残果、芽、落叶和杂草内未发现越冬茧;各部位的梨小食心虫越冬茧比例分别为:地上部占越冬茧总量的89.62%,土壤中占10.31%,地表枯枝中占0.69%。[结论]为梨小食心虫越冬代的防治提供了理论依据。     

Detection and enumeration of wheat grains based on a deep learning method under various scenarios and scales

Grain number is crucial for analysis of yield components and assessment of effects of cultivation measures. The grain number per spike and thousand-grain weight can be measured by counting grains manually, but it is time-consuming, tedious and error-prone. Previous image processing algorithms cannot work well with different backgrounds and different sizes. This study used deep learning methods to resolve the limitations of traditional image processing algorithms. Wheat grain image datasets were collected in the scenarios of three varieties, six background and two image acquisition devices with different heights, angles and grain numbers, 1 748 images in total. All images were processed through color space conversion, image flipping and rotation. The grain was manually annotated, and the datasets were divided into training set, validation set and test set. We used the TensorFlow framework to construct the Faster Region-based Convolutional Neural Network Model. Using the transfer learning method, we optimized the wheat grain detection and enumeration model. The total loss of the model was less than 0.5 and the mean average precision was 0.91. Compared with previous grain counting algorithms, the grain counting error rate of this model was less than 3% and the running time was less than 2 s. The model can be effectively applied under a variety of backgrounds, image sizes, grain sizes, shooting angles, and shooting heights, as well as different levels of grain crowding. It constitutes an effective detection and enumeration tool for wheat grain. This study provides a reference for further grain testing and enumeration applications.     

遮光性套袋对桃果实转录组的影响

【目的】探明遮光性套袋在桃果实上的转录组差异,丰富桃转录组数据信息。【方法】选取遮光性套袋与对照的桃果实样品,利用Illumina Hi Seq TM 2500进行高通量测序,构建桃果实转录组文库,并用测序评估、基因功能注释等生物信息学方法进行分析。【结果】经过测序获得16.62 Gb clean data测序数据,且碱基百分比(Q30)大于91%,遮光性套袋和对照2个桃果实样品分别获得65 300 730个reads和66 603 686个reads,分别有85.73%和84.60%的reads与桃参考基因组匹配。以无袋处理为参考,对转录组数据进行比较,遮光性套袋处理共获得1 963个差异表达基因,其中,下调基因708个,上调基因1 255个;在Nr数据库中对差异基因进一步进行注释,注释到1 957个基因,其中,下调基因有705个,上调基因有1 252个;COG功能注释分析发现这些差异表达基因共获得853个功能注释,涉及23个功能类别;在GO功能注释分析中,注释到1 609个基因,可以分为53个功能分类,这些分类主要涉及到分子结合、催化活性、细胞过程、生物调节等诸多生理生化过程;KEGG分析发现共有421个基因被注释到94个代谢通路中,其中,光合作用相关通路、类黄酮生物合成、核糖体生物合成等通路显著富集。光合作用通路和类黄酮生物合成通路在果实着色中发挥了重要作用,而核糖体生物合成代谢通路在果实成熟着色中的作用尚不明确。同时也进行了两组样品的果实品质检测,结果表明,遮光性套袋对桃果实的可溶性固形物及可溶性总糖产生显著性影响,可溶性固形物及可溶性总糖显著降低,而对于果实总酸的影响不大,在果实大小上几乎没有影响。【结论】在遮光性套袋处理状态下,获得一定数量的桃果实差异表达基因,光合作用通路和类黄酮生物合成通路基因在果实着色中发挥重要作用,遮光性套袋对桃果实的可溶性固形物及可溶性总糖产生显著性影响。     

Comprehensive evaluation of 20 pomegranate (Punica granatum L.) cultivars in China

Recent investigations on pomegranate products have significantly increased and successfully drawn consumers’ attention to nutritional and medicinal values, promoting the pomegranate industry's development worldwide. However, little information on pomegranates grown in China is available. Morphological and chemical characterizations of fruits and arils from 20 pomegranate cultivars in six regions of China were investigated. Combined with overall scores by principal component analysis, ‘Yushiliu No. 1’, ‘Taishanhong No. 2’, ‘Tunisia’ and ‘Mollar’ were promising cultivars, and Chinese researchers bred the first two. It was surprising that ‘Mollar’ had bigger fruit size and more aril moisture grown in China than in Spain. Cultivars with higher anthocyanin content in arils were ‘Turkey’, ‘Moyu’ and ‘Red Angel’, which might be used as the source of natural red food colourants. While red husk ‘Hongruyi’ and ‘Hongshuangxi’ with higher vitamin C, aril moisture and lower titratable acid in arils, might also be promising cultivars for further various utilization. Furthermore, the comparison of ‘Tunisia’ fruits from four regions revealed that cultivation locations had more influence on fruit traits than genotypes. Maturity index classification was established for Chinese pomegranate cultivars. Therefore, the results would provide a valuable guide for agricultural cultivation, industrial utilization, and breeding.     

肥城桃果实不同发育时期的香气组分及其变化

以21年生‘白里’肥城桃为研究材料,运用气–质联用技术(GC–MS),对肥城桃果实绿熟期、白熟期和完熟期的香气组分及其含量变化进行研究。结果表明,在果实中共检测到63种香气成分,这些香气物质主要为醛类、醇类、酯类和内酯类化合物。醛类物质主要为C6醛类和芳香醛类化合物;醇类物质主要为C6醇类和C5醇类化合物,芳香醇类化合物含量极少,C6醇类化合物含量随果实成熟逐渐降低。随果实的成熟,酯类物质的含量迅速上升,这主要是由乙酸乙酯含量增加所致。γ–己内酯、γ–庚内酯、δ–辛内酯仅在白熟期和完熟期能检测到。己醛、(Z)–3–己烯醛、(E,E)–2,4–己二烯醛、2–环己烯–1–醇是未成熟果实的特征香气成分;(E)–2–己烯醛、乙酸乙酯、γ–己内酯、γ–庚内酯、δ–辛内酯是成熟果实的特征香气成分。     

Passion fruit detection and counting based on multiple scale faster R-CNN using RGB-D images

The accurate and reliable fruit detection in orchards is one of the most crucial tasks for supporting higher level agriculture tasks such as yield mapping and robotic harvesting. However, detecting and counting small fruit is a very challenging task under variable lighting conditions, low-resolutions and heavy occlusion by neighboring fruits or foliage. To robustly detect small fruits, an improved method is proposed based on multiple scale faster region-based convolutional neural networks (MS-FRCNN) approach using the color and depth images acquired with an RGB-D camera. The architecture of MS-FRCNN is improved to detect lower-level features by incorporating feature maps from shallower convolution feature maps for regions of interest (ROI) pooling. The detection framework consists of three phases. Firstly, multiple scale feature extractors are used to extract low and high features from RGB and depth images respectively. Then, RGB-detector and depth-detector are trained separately using MS-FRCNN. Finally, late fusion methods are explored for combining the RGB and depth detector. The detection framework was demonstrated and evaluated on two datasets that include passion fruit images under variable illumination conditions and occlusion. Compared with the faster R-CNN detector of RGB-D images, the recall, the precision and F1-score of MS-FRCNN method increased from 0.922 to 0.962, 0.850 to 0.931 and 0.885 to 0.946, respectively. Furthermore, the MS-FRCNN method effectively improves small passion fruit detection by achieving 0.909 of the F1 score. It is concluded that the detector based on MS-FRCNN can be applied practically in the actual orchard environment.