Detection and counting of immature green citrus fruit based on the Local Binary Patterns (LBP) feature using illumination-normalized images

Early detection and counting of immature green citrus fruit using computer vision can help growers produce a predictive yield map which could be used to adjust management practices during the fruit maturing stages. However, such detecting and counting is difficult because of varying illumination, random occlusion and color similarity with leaves. An immature fruit detection algorithm was developed with the aim of identifying and counting fruit in a citrus grove under varying illumination environments and random occlusions using images acquired by a regular red–green–blue (RGB) color camera. Acquired citrus images included front-lighting and back-lighting illumination conditions. The Retinex image enhancement algorithm and the two-dimensional discrete wavelet transform were used for image illumination normalization. Color-based K-means clustering and circular hough transform (CHT) were applied in order to detect potential fruit regions. A Local Binary Patterns feature-based Adaptive Boosting (AdaBoost) classifier was built for removing false positives. A sub-window was used to scan the difference image between the illumination-normalized image and the resulting image from CHT detection in order to detect small areas and partially occluded fruit. An overall accuracy of 85.6% was achieved for the validation set which showed promising potential for the proposed method.     

Citrus fruits maturity detection in natural environments based on convolutional neural networks and visual saliency map

Citrus fruits do not ripen at the same time in natural environments and exhibit different maturity stages on trees, hence it is necessary to realize selective harvesting of citrus picking robots. The visual attention mechanism reveals a physiological phenomenon that human eyes usually focus on a region that is salient from its surround. The degree to which a region contrasts with its surround is called visual saliency. This study proposes a novel citrus fruit maturity method combining visual saliency and convolutional neural networks to identify three maturity levels of citrus fruits. The proposed method is divided into two stages: the detection of citrus fruits on trees and the detection of fruit maturity. In stage one, the object detection network YOLOv5 was used to identify the citrus fruits in the image. In stage two, a visual saliency detection algorithm was improved and generated saliency maps of the fruits; The information of RGB images and the saliency maps were combined to determine the fruit maturity class using 4-channel ResNet34 network. The comparison experiments were conducted around the proposed method and the common RGB-based machine learning and deep learning methods. The experimental results show that the proposed method yields an accuracy of 95.07%, which is higher than the best RGB-based CNN model, VGG16, and the best machine learning model, KNN, about 3.14% and 18.24%, respectively. The results prove the validity of the proposed fruit maturity detection method and that this work can provide technical support for intelligent visual detection of selective harvesting robots.

     

Immature peach detection in colour images acquired in natural illumination conditions using statistical classifiers and neural network

Detection of immature peach fruits would help growers to create yield maps which are very useful tools for adjusting management practices during the fruit maturing stages. Machine vision algorithms were developed to detect and count immature peach fruit in natural canopies using colour images. This study was the first effort to detect immature peach fruit in natural environment to the authors’ knowledge. Captured images had various illumination conditions due to both direct sunlight and diffusive light conditions that make the fruit detection task more difficult. A training set and a validation set were used to develop and to test the algorithms. Different image scanning methods including finding potential fruit regions were developed and used to parse fruit objects in the natural canopy image. Circular Gabor texture analysis and ‘eigenfruit’ approach (inspired by the ‘eigenface’ face detection and recognition method) were used for feature extraction. Statistical classifiers, a neural network and a support vector machine classifier were built and used for detecting peach fruit. A blob analysis was performed to merge multiple detections for the same peach fruit. Performance of the classifiers and image scanning methods were introduced and evaluated. Using the proposed algorithms, 84.6, 77.9 and 71.2 % of the actual fruits were successfully detected using three different image scanning methods for the validation set.     

一种基于信息融合苹果果实世界定位方法

构建由Kinect设备、差分GPS设备和陀螺仪组成的信息获取系统,推导RGB图像中苹果果实世界坐标的计算方法。利用Kinect设备获取果园果树RGB图像和深度图像,差分GPS全球定位系统和陀螺仪分别获取Kinect相机位置信息和姿态信息。融合RGB图像和深度图像,利用相对位置定位模型,计算苹果圆心相机空间坐标,融合相机位置信息和姿态信息,利用空间三维坐标转换原理,建立绝对位置定位模型计算苹果世界坐标,对每个果实进行世界空间位置唯一标定。结果表明,果实相对平均定位误差0.035 m,果实绝对定位经度误差0.117 m,纬度误差0.437 m,海拔误差0.145 m。     

自然环境下果实作业机器人幼果期苹果侦测方法

首先,采用自适应G-B色差法对初始图像计算,获得色差灰度图,使用迭代阈值分割法提取果实兴趣区;其次,对经形态学处理后的兴趣区图像进行Blob分析,计算每个Blob的离心率和像素面积,去除明显偏离果实形状特点的Blob;最后,应用改进圆形Hough变换算法检测潜在类圆形果实目标,最终采用融合方向梯度直方图特征和网格搜索优化支持向量机的判别模型进一步去除虚假果实目标,提升苹果目标的侦测精确度。试验结果显示,该方法对果园自然环境下幼小青苹果的侦测正确率为88.51%,漏报率和误报率分别为11.49%和4.84%,算法模型综合性能指标为90.29%,表明该方法对幼果期苹果目标具有较强的侦测能力和较好的鲁棒性,该结果为果实作业机器人幼果期的自动化果实侦测提供参考。     

成熟期苹果计数方法研究———以黄土高原样本为例证

对比RGB(红,绿,蓝)颜色空间下各颜色分量间多种色差运算的自适应阈值分割与基于H(色调)和S(饱和度)的K均值聚类算法对苹果影像的分割,选取适宜于自然生长状态下成熟期苹果影像分割的最佳算法分割目标物.在影像分割的基础之上,通过计算影像中苹果区域的总面积与单果平均面积之商确定苹果数目.试验结果表明:1.1×R-G色差运算结合自适应阈值分割算法对成熟期苹果影像有较好的分割效果;以影像中苹果总面积与单果平均面积之商确定苹果数目的算法准确率可达82.18%,计数方法准确率高.     

基于高斯混合模型的类人机器人果实辨识研究

针对类人机器人的特点,设计了一个具有果实辨识功能的类人机器人系统。类人机器人自带的摄像头作为视觉系统,通过视觉系统收集彩色果实图片,将RGB颜色转换成HSV颜色,结合高斯混合模型算法,通过训练得到每类果实所对应的分类器模型参数,构造分类器,实现类人机器人对果实准确高效的识别。     

基于彩色点云图像的不同成熟阶段番茄果实数量的测定方法

【目的】为测定温室中番茄不同成熟阶段的果实数量,提出一种基于彩色点云图像的测定方法。【方法】在移动平台上搭载KinectV2.0采集温室中行栽番茄的图像信息合成番茄植株点云,再将二视角的番茄植株点云合成1个点云,并通过深度信息截取得到近处番茄植株点云,将标注的点云数据输入到PointRCNN目标检测网络训练预测模型,并识别番茄植株点云中的番茄果实,最后利用基于特征矩阵训练的支持向量机(Support vector machine, SVM)分类器对已经识别出来的果实进行成熟阶段分类,获得不同成熟阶段番茄果实的数量。【结果】基于PointRCNN目标检测网络的方法识别番茄果实数量的精确率为86.19%,召回率为83.39%;基于特征矩阵训练的SVM分类器,针对番茄果实成熟阶段的预测结果在训练集上准确率为94.27%,测试集上准确率为96.09%。【结论】基于彩色点云图像的测定方法能够较为准确地识别不同成熟阶段的番茄果实,可以为评估温室番茄产量提供数据支撑。     

广东省百香果产业现状与发展对策

百香果是著名的果汁型热带水果,集高营养价值、观赏价值、药用价值于一身。百香果产业是近年广东省大力发展的新兴产业,具有广阔的市场前景。分析了广东省百香果的产业现状及主要存在的问题,包括优质种质资源缺乏,健康种苗繁育技术溃乏;高效高产能配套栽培技术尚不完善;病虫害及逆境灾害防范不足;产、销、加工产业链不完善;产业发展科技支撑及政策扶持不足等。这些产业问题制约了广东百香果的综合利用和发展,亟需解决。针对百香果产业问题提出了广东省百香果产业发展的相应策略,包括加强百香果品种资源收集与利用;加强百香果良种良法种植;加强广东省百香果产业创新联盟构建,产销形式多元化发展;发挥科技支撑作用,加强培养新型经营主体;加强百香果产业政策扶持与管理。     

Low and high-level visual feature-based apple detection from multi-modal images

Automated harvesting requires accurate detection and recognition of the fruit within a tree canopy in real-time in uncontrolled environments. However, occlusion, variable illumination, variable appearance and texture make this task a complex challenge. Our research discusses the development of a machine vision system, capable of recognizing occluded green apples within a tree canopy. This involves the detection of “green” apples within scenes of “green leaves”, shadow patterns, branches and other objects found in natural tree canopies. The system uses both thermal infra-red and color image modalities in order to achieve improved performance. Maximization of mutual information is used to find the optimal registration parameters between images from the two modalities. We use two approaches for apple detection based on low and high-level visual features. High-level features are global attributes captured by image processing operations, while low-level features are strong responses to primitive parts-based filters (such as Haar wavelets). These features are then applied separately to color and thermal infra-red images to detect apples from the background. These two approaches are compared and it is shown that the low-level feature-based approach is superior (74% recognition accuracy) over the high-level visual feature approach (53.16% recognition accuracy). Finally, a voting scheme is used to improve the detection results, which drops the false alarms with little effect on the recognition rate. The resulting classifiers acting independently can partially recognize the on-tree apples, however, when combined the recognition accuracy is increased.     

金柑果实生长发育的数学模型研究

[目的]建立金柑果实生长发育的数学模型,以确定适合金柑生长的栽培措施。[方法]以融安金柑为试材,通过测定金柑果实生长发育期间果实纵径、横径、发育天数等指标,建立融安金柑果实的生长模型,明确其相互间的变化规律。[结果]花后30 d内,金柑果实的纵、横径存在1个迅速生长期,期间果实纵径发育速度明显快于横径;花后30 d后,果实发育进入缓慢生长期,果实横径发育速度略快于纵径;花后100～110 d,果实大小有1个增长小高峰。果实横径（y）与发育天数（x）之间的生长模型方程为y=0.000 057x2-0.007 971x＋0.611 333,R2=0.995 0;果实纵径（y）与发育天数（x）之间的生长模型方程为y=0.000 097x2-0.013 264x＋0.855 225,R2=0.990 2。[结论]金柑果实横径、纵径与发育天数之间存在明显的多项式回归关系,且其生长进程数学模型同为二次方程。     

基于深度视觉和蚁群避障算法的果园精准喷施路径规划方法

为了实现对Y-shaped果树的精准喷施,本文融合了彩色及深度视觉图像,提出了1种基于蚁群避障算法的果园最优行驶路径的规划方法。首先,对彩色图像进行图像分割处理,划分道路及果树树墙障碍区域,得出喷施设备的可行驶区域,提出了喷施行驶范围检测算法;然后,通过对深度图像和彩色图像融合的处理,将Y-shaped果树树冠边缘轮廓精准拟合形成栅格地图,并与蚁群避障算法相结合,提出了最优行驶路径规划算法。最后,对拟合曲线和Y-shaped果树树冠边缘轮廓进行检验,验证算法的拟合程度。实验结果证明,本文提出的路径规划算法可以准确地检测出果树区域,并实现对行驶路径的精准规划。     

固定化酵母发酵木瓜-西番莲复合果酒工艺的研究

[目的]研究固定化酵母发酵木瓜-西番莲复合果酒的最佳工艺条件。[方法]以木瓜和西番莲果为原料,研究了初始加糖量、pH、固定化酵母接种量、发酵温度对木瓜-西番莲复合果酒残糖、酒精度和感官品质的影响。[结果]影响木瓜西番莲复合果酒发酵残糖、酒精度和感官品质的主要因素为复合果汁初始加糖量,最佳发酵条件为混合果汁起始糖度28%、pH 4.5、酵母接种量0.015%、发酵温度25℃。[结论]在最佳工艺条件下可获得色泽淡黄自然、拥有着和谐的果香与酒香、入口清爽、酒精含量约13.8%的木瓜-西番莲复合果酒。     

基于数字图像的水果分形维数特征

为了定量描述仿球形水果表面轮廓间的差异,用周长-面积法研究水果的分形维数特征.取大小不同的温州蜜柑20个、麻阳冰糖橙17个和红富士苹果18个,设置3种水果花萼面和侧面2个方向上数字图像红色或蓝色分量阈值,将水果区域内外的红色、绿色、蓝色分量分别置为0和1,得到水果二值图像.以水果区域细化后的边界像素数和区域内像素数分别作为水果的周长和面积,以周长-面积法求得3类水果2个方向的分形维数.结果表明,3种水果花萼面的分形维数均比其侧面的分形维数小,花萼面分形维数间差异不明显,但侧面分形维数差异明显,表明仿球形水果的外形特征可用花萼面分形维数和侧面分形维数反映,但主要指标是侧面分形维数.     

复杂背景下油茶果采收机重叠果实定位方法研究

油茶果机械化振动采摘技术关键在于振动点选取,判断振动点选取取决于果实生长密度测算和分布估计.然而自然环境下重叠果实的识别对判定结果有较大的影响,因此提出一种基于凸壳识别的分割边界优化方法,提升重叠油茶果识别与分割准确度.该方法先将原始图像转换颜色空间,经过阈值分割和形态学处理获得重叠果实的凹区域,然后在此基础上通过Harris角点检测得到区域的特征点集,利用主成分分析(PCA)和欧式距离方法分析特征点距离关系得到分割路径,最后采用最小二乘法对分割后的目标区域进行拟合重建得到果实轮廓.对比重建的果实轮廓与真实分布图像,该方法的平均定位误差为8.6％,比Hough方法低5.1％;平均耗时为0.52 s,比Hough方法低0.12 s.结果 表明,提出的方法可以有效解决重叠油茶果实识别与分割问题,为采摘装置的振动点选择奠定基础.     

Green citrus detection using ‘eigenfruit’, color and circular Gabor texture features under natural outdoor conditions

A machine vision algorithm was developed to detect and count immature green citrus fruits in natural canopies using color images. A total of 96 images were acquired in October 2010 from an experimental citrus grove in the University of Florida, Gainesville, Florida. Thirty-two of the total 96 images were selected randomly and used for training the algorithm, and 64 images were used for validation. Color, circular Gabor texture analysis and a novel ‘eigenfruit’ approach (inspired by the ‘eigenface’ face detection and recognition method) were used for green citrus detection. A shifting sub-window at three different scales was used to scan the entire image for finding the green fruits. Each sub-window was classified three times by eigenfruit approach using intensity component, eigenfruit approach using saturation component, and circular Gabor texture. Majority voting was performed to determine the results of the sub-window classifiers. Blob analysis was performed to merge multiple detections for the same fruit. For the validation set, 75.3% of the actual fruits were successfully detected using the proposed algorithm.     

Immature green citrus fruit detection and counting based on fast normalized cross correlation (FNCC) using natural outdoor colour images

A fast normalized cross correlation (FNCC) based machine vision algorithm was proposed in this study to develop a method for detecting and counting immature green citrus fruit using outdoor colour images toward the development of an early yield mapping system. As a template matching method, FNCC was used to detect potential fruit areas in the image, which was the very basis for subsequent false positive removal. Multiple features, including colour, shape and texture features, were combined in this algorithm to remove false positives. Circular Hough transform (CHT) was used to detect circles from images after background removal based on colour components. After building disks centred in centroids resulted from both FNCC and CHT, the detection results were merged based on the size and Euclidian distance of the intersection areas of the disks from these two methods. Finally, the number of fruit was determined after false positive removal using texture features. For a validation dataset of 59 images, 84.4 % of the fruits were successfully detected, which indicated the potential of the proposed method toward the development of an early yield mapping system.     

紫果西番莲开花结果习性的观察研究

紫果西番莲具有一年多次开花结果的习性,主要靠春梢结果。福建中部地区开花期在4月上旬至5月中旬,一朵花开放的时间为1～2天。花蕾大部分着生于结果蔓的中部或中上部,自然座果率为23.5%,但在盛花期进行人工授粉可使座果率提高到36.5%。开花后15天果实已增大到接近最大程度,18天后一般不再增大,而集中于种子的发育和可食部分假种皮的形成,果实从受精至完熟历时89～91天。本文还阐述了影响开花座果的有关因子,并提出了重施花前肥、适当修剪和花期放蜂等提高座果率的有效措施。     

Machine vision for counting fruit on mango tree canopies

Machine vision technologies hold the promise of enabling rapid and accurate fruit crop yield predictions in the field. The key to fulfilling this promise is accurate segmentation and detection of fruit in images of tree canopies. This paper proposes two new methods for automated counting of fruit in images of mango tree canopies, one using texture-based dense segmentation and one using shape-based fruit detection, and compares the use of these methods relative to existing techniques:—(i) a method based on K-nearest neighbour pixel classification and contour segmentation, and (ii) a method based on super-pixel over-segmentation and classification using support vector machines. The robustness of each algorithm was tested on multiple sets of images of mango trees acquired over a period of 3 years. These image sets were acquired under varying conditions (light and exposure), distance to the tree, average number of fruit on the tree, orchard and season. For images collected under the same conditions as the calibration images, estimated fruit numbers were within 16 % of actual fruit numbers, and the F1 measure of detection performance was above 0.68 for these methods. Results were poorer when models were used for estimating fruit numbers in trees of different canopy shape and when different imaging conditions were used. For fruit-background segmentation, K-nearest neighbour pixel classification based on colour and smoothness or pixel classification based on super-pixel over-segmentation, clustering of dense scale invariant feature transform features into visual words and bag-of-visual-word super-pixel classification using support vector machines was more effective than simple contrast and colour based segmentation. Pixel classification was best followed by fruit detection using an elliptical shape model or blob detection using colour filtering and morphological image processing techniques. Method results were also compared using precision–recall plots. Imaging at night under artificial illumination with careful attention to maintaining constant illumination conditions is highly recommended.     

Fusion of the YOLOv4 network model and visual attention mechanism to detect low-quality young apples in a complex environment

Precision Agriculture - The accurate detection of young fruits in complex scenes is of great significance for automatic fruit growth monitoring systems. The images obtained in the open orchard...