Registration of multispectral 3D points for plant inspection

Machine vision technologies have shown advantages for efficient and accurate plant inspection in precision agriculture. Regarding the balance between accuracy of inspection and compactness for infield applications, multispectral imaging systems would be more suitable than RGB colour cameras or hyperspectral imaging systems. Multispectral image registration (MIR) is a key issue for multispectral imaging systems, however, this task is challenging. First of all, in many cases, two images needing registration do not have a one-to-one linear mapping in 2D space and therefore they cannot be aligned in 2D images. Furthermore, the general MIR algorithms are limited to images with uniform intensity and are incapable of registering images with rich features. This study developed a machine vision system (MVS) and a MIR method which replaces 2D-2D image registration by 3D-3D point cloud registration. The system can register 3D point clouds of ultraviolet (UV), blue, green, red and near-infrared (NIR) spectra in 3D space. It was found that the point clouds of general plants created by images of different spectral bands have a complementary property, and therefore a combined point cloud, called multispectral 3D point cloud, is denser than any cloud created by a single spectral band. Intensity information of each spectral band is available in a multispectral 3D point cloud and therefore image fusion and 3D morphological analysis can be conducted in the cloud. The MVS could be used as a sensor of a robotic system to fulfil on-the-go infield plant inspection tasks.     

基于深度学习的农作物病害图像识别技术进展

农作物病害的无损检测和早期识别是精准农业和生态农业发展的关键。随着图像采集和图像处理技术的进步,高光谱成像等先进成像探测技术和基于深度学习的图像分析技术越来越多地应用于农作物病虫害的无损检测中。本文首先简单介绍了以深度学习为代表的图像识别技术的基本原理,然后系统地阐述了基于深度学习的先进成像技术和先进图像识别分析技术在农作物病害检测识别中的国内外研究现状,分析了其在农作物病害检测识别上存在的优缺点,如具有快速、准确率高等优点以及数据量过大处理不便等缺点,并进一步指出,利用高光谱成像和热红外成像与深度学习相结合,将成为今后研究农作物病虫害早期检测的主要发展方向。     

基于高光谱图像的新疆红富士苹果颜色分级研究

[目的]以新疆红富士苹果为研究对象,探讨应用高光谱图像技术对其着色面积进行的研究方法.[方法]对852/713双波段比图像作阈值分割,以及形态学开运算去除果梗区域,提取色调H灰度图像对应去除果梗的二值图像像素值为1的累计频度值,依据AdaBoost算法将15个BP神经网络弱分类器训练组成强分类器,对苹果的着色面积进行分类.[结果]采用AdaBoost_NN对苹果着色面积的分级与人工分级一致率达到97.7;.其中45个优等果有2个被错分为一等果,27个等外果有1个被错分为二等果.[结论]利用高光谱图像技术提取的特征波长图像能够很好的对苹果着色面积进行分级,为今后多光谱成像技术在线分析苹果品质奠定研究基础.     

多旋翼无人机载高光谱成像系统几何和辐射校正方法研究

针对农林环保等行业应用,结合成像光谱仪与无人机(unmanned aerial vehicle, UAV)技术,有效地解决了高光谱遥感数据不足和空间分辨率低等问题.但UAV高光谱成像系统面临的设备昂贵、数据质量差和效率低等一系列问题,致使低成本高效便捷的UAV高光谱成像系统成为了研究的关键.介绍了将室内成像光谱仪搭载在多旋翼UAV上采集数据的高光谱成像系统,在试验区布设控制点和标准灰布,用GPS-RTK测量控制点的三维坐标和ASD地物光谱仪测量标准灰布的反射率以验证系统的成像精度.从辐射和几何两个方面进行分析评价.在辐射方面,经过辐射定标和MODTRAN模型大气校正的高光谱数据中校准灰布的反射率与ASD地物光谱仪测量的结果十分吻合;在几何方面,用GPS-IMU数据进行初始几何校正的图像与地面控制点坐标以分析图像处理后的几何误差,并提出通过改正姿态偏转角的方法提高图像的地理位置精度.     

Automatic X-ray quarantine scanner and pest infestation detector for agricultural products

This paper presents a new automatic and effective quarantine system for detecting pest infestation sites in agricultural products, e.g. fruits. This work integrated mechanical design, mechatronics instrumentation, X-ray and charge-coupled device (CCD) image acquisition devices, LabVIEW-based analysis and control software, and image diagnosis algorithms into the automatic X-ray quarantine scanner system. Based on the LabVIEW development platform, a friendly graphical user interface (GUI) was designed for assisting the operations of quarantine scanner system. To enhance the accuracy and efficiency of pest quarantine process, a control scheme for performing start-up procedure of the system, parameter setting and calibration of the X-ray source and line-scan sensor, and automatic inspection for pest were developed. A novel pest infestation detector consisted of image processing algorithms were also proposed to aid the operator in identifying possibly infested fruits. The image processing procedures include contrast enhancement, median filtering, mathematical morphology operators, and adaptive thresholding by statistical z-test for identifying the infested sites of fruit on an X-ray image. Experimental results show that the X-ray quarantine scanner and pest infestation detector are able to locate the infested sites with highly successful rate up to 94% on the 4th day after eggs implanted. Furthermore, both intact and egg-implanted fruits were used to evaluate the sensitivity, specificity, accuracy, and precision of the proposed system. The evaluation results are respectively 96.8%, 98.6%, 97.7%, and 98.7%, which are significantly better than traditional visual inspection.     

Detection of contaminated hazelnuts and ground red chili pepper flakes by multispectral imaging

Mycotoxins are the toxic metabolites of certain filamentous fungi and have been demonstrated to cause various health problems in humans, including immunosuppression and cancer. Among them, the aflatoxins have received greater attention because they are potent carcinogens and are responsible for many human deaths per annum, mostly in non-industrialized countries. Various regulatory agencies have enforced limits on the concentrations of these toxins in foods and feeds involved in international commerce. Hyperspectral and multispectral imaging are becoming increasingly important for rapid and nondestructive testing for the presence of such contaminants. However, the high number of spectral bands needed may render such image acquisition systems too complex, expensive and slow. Moreover, they tend to generate overwhelming amount of data, making effective processing of this information in real time difficult. In this study, a two-dimensional local discriminant bases algorithm was developed to detect the location of the discriminative features in the multispectral data space. The algorithm identifies the optimal passband width and center frequencies of optical filters to be used for a multispectral imaging system. This was applied to a multispectral imaging system used to detect aflatoxin-contaminated hazelnut kernels and red chili peppers. Classification accuracies of 92.3% and 80% were achieved for aflatoxin-contaminated and uncontaminated hazelnuts and red chili peppers, respectively. The aflatoxin concentrations were decreased from 608 to 0.84 ppb for tested hazelnuts and from 38.26 to 22.85 ppb for red chili peppers by removal of the nuts/peppers that were classified as aflatoxin-contaminated. The algorithm was also used to classify fungal contaminated and uncontaminated hazelnut kernels, and an accuracy of 95.6% was achieved for this broader classification.     

基于光谱技术的禽蛋内部品质无损检测研究进展

禽蛋品质检测是食品安全和消费者权益的重要保障措施,传统禽蛋品质检测主要依赖人工进行,存在工作强度大、效率低且准确率波动大等弊端。光谱检测技术具有快速、安全、无损等优点,近些年来在禽蛋内部品质检测领域发展迅速。本文基于禽蛋的新鲜度、蛋白含量、脂肪含量、血斑肉斑、受精信息、种蛋性别、胚蛋活性等内部品质指标检测的有关研究,概述了近红外光谱、可见-近红外光谱、高光谱成像及拉曼光谱等光谱检测技术在禽蛋内部品质无损检测中的研究进展,分析总结了光谱检测技术在禽蛋无损检测中的应用特点与难点,并展望了其未来发展趋势,以期为我国蛋品无损检测研究及行业质量安全监管提供参考。     

Development of software for spectral imaging data acquisition using LabVIEW

Developing data acquisition software is a major challenge in integrating a spectral imaging system. This paper presents the design and implementation of a data acquisition program using LabVIEW for a liquid crystal tunable filter based spectral imaging system (900–1700 nm). The module-based program was designed in a three-tier structure. The image acquisition process, modelled by a finite state machine, was implemented in LabVIEW to control the spectral imaging system to collect hyperspectral or multispectral images. The collected spectral images were encoded in general format and could be further processed by other common spectral image analysis tools. In addition, the program could be used to observe band ratio images of the test object in real-time, collect spectral images after ensemble averaging, and select region of interest for spectral image acquisitions. This program is a useful data acquisition tool for the filter-based spectral imaging system. The design and implementation techniques described in this article could also be used to develop similar spectral image acquisition programs.     

Detection of common defects on oranges using hyperspectral reflectance imaging

To detect various common defects on oranges, a hyperspectral imaging system has been built for acquiring reflectance images from orange samples in the spectral region between 400 and 1000 nm. Oranges with insect damage, wind scarring, thrips scarring, scale infestation, canker spot, copper burn, phytotoxicity, heterochromatic stripe, and normal surface were studied. Hyperspectral images of samples were evaluated using principal component analysis (PCA) with the goal of selecting several wavelengths that could potentially be used in an in-line multispectral imaging system. The third principal component images using six wavelengths (630, 691, 769, 786, 810 and 875 nm) in the visible spectral (VIS) and near-infrared (NIR) regions, or the second principal component images using two wavelengths (691 and 769 nm) in VIS region gave better identification results under investigation. However, the stem-ends were easily confused with defective areas. In order to solve this problem, representative regions of interest (ROIs) reflectance spectra of samples with different types of skin conditions were visually analyzed. The researches revealed that a two-band ratio (R875/R691) image could be used to differentiate stem-ends from defects effectively. Finally, the detection algorithm of defects was developed based on PCA and band ratio coupled with a simple thresholding method. For the investigated independent test samples, accuracies of 91.5% and 93.7% with no false positives were achieved for both sets of selected wavelengths using proposed method, respectively. The disadvantage of this algorithm is that it could not discriminate between different types of defects.     

玉米籽粒性状高通量测量装置设计与实现

为实现玉米籽粒表型性状参数的高通量测量,设计了一种基于线扫描技术和自动化控制技术相结合的玉米籽粒考种装置。该系统通过振动给料机实现玉米籽粒快速喂料,应用伺服驱动技术实现输送带运行速度和线阵扫描速度无偏差匹配,实现玉米籽粒图像无畸变获取,通过图像处理技术实现玉米籽粒表型性状参数的测量。试验结果表明,与人工测量值比较,该装置对玉米籽粒总粒数、长轴、短轴、长宽比的平均相对误差分别为0.50%、1.22%、3.34%、4.22%,平均测量效率为12 s/穗。该研究对于推动玉米籽粒高通量表型鉴定具有重要意义。     

基于近红外高光谱图像技术的栗果品质无损检测

提出一种基于近红外高光谱图像技术的板栗果实品质快速无损检测方法。分别选取3个不同品种栗果、1个品种的霉变栗果和1个品种的虫害栗果各30个样品,采集供试样品的近红外高光谱数据;采用偏最小二乘法(PLS)建立栗果中总糖和淀粉含量预测模型,预测值与实际值的相关系数为0.9313～0.9587,均方根误差为0.062 4～0.225 0;结合主成分分析法(PCA),建立不同品种栗果鉴别以及识别霉变、虫害、正常栗果的判别分析(DA)模型,模型的识别率分别为96.7%和98.6%。结果表明,近红外高光谱图像技术可用于栗果总糖和淀粉的定量预测,以及不同品种栗果和霉变、虫害果的快速定性识别。     

Feature extraction of hyperspectral images for detecting immature green citrus fruit

At an early immature growth stage of citrus, a hyperspectral camera of 369–1042 nm was employed to acquire 30 hyperspectral images in order to detect immature green fruit within citrus trees under natural illumination conditions. First, successive projections algorithm (SPA) were implemented to select 677, 804, 563, 962, and 405 nm wavebands and to construct multispectral images from the original hyperspectral images for further processing. Then, histogram threshold segmentation using NDVI of 804 and 677 nm was implemented to remove image backgrounds. Three slope parameters, calculated from the pairs 405 and 563 nm, 563 and 677 nm, and 804 and 962 nm were used to construct a classifier to identify the potential citrus fruit. Then, a marker-controlled watershed segmentation based on wavelet transform was applied to obtain potential fruit areas. Finally, a green fruit detection model was constructed according to Grey Level Co-occurrence Matrix (GLCM) texture features of the independent areas. Three supervised classifiers, logistic regression, random forest and support vector machine (SVM) were developed using texture features. The detection accuracies were 79%, 75%, and 86% for the logistic regression, random forest, and SVM models, respectively. The developed algorithm showed a great potential for identifying immature green citrus for an early yield estimation.     

高光谱成像技术在水果品质无损检测中的研究进展

优质水果的生产和销售离不开水果品质检测,传统的水果品质检测手段精度低、成本高、时效性差、破坏性强。近年来,随着科学技术的不断进步,低成本、高效率的水果品质无损检测技术得到飞速发展。其中,高光谱成像技术逐渐成为研究热点。综述了该技术在水果品质无损检测方面的技术原理、应用和发展现状,探讨其在水果品质无损检测领域的应用潜力、存在问题、发展趋势以及应用前景。整体来看,高光谱成像技术能够实现不同水果种类、多个水果品质指标的无损、高效检测,如成熟度、糖度、酸度、红色指数等;受硬件技术限制,其发展侧重于数据挖掘方向,即在硬件发展有限的情况下,通过不断更新和优化的针对性算法获得精准的解析结果;另一方面,设备昂贵、数据处理复杂、模型普适性较差是该技术需要进一步优化和改进的主要问题;其未来发展将基于云计算和人工智能的高效数据处理、适用范围更广的水果品质高光谱检测设备研发、多源综合无损检测等研究方向。随着技术的不断发展,高光谱成像技术在水果品质无损检测方面的应用前景广阔,未来将成为水果品质检测的重要手段之一。     

基于高光谱成像的猕猴桃表面疤痕无损识别

为实现猕猴桃表面疤痕的快速无损识别,以贵长猕猴桃为研究对象,采用高光谱图像采集系统(400~1 000 nm)采集完好无损猕猴桃和表面有疤痕猕猴桃的高光谱图像。对采集到的高光谱图像进行了最小噪声分离变换,结合阈值分割及数学形态学处理方法提出了猕猴桃表面疤痕的识别方法。结果表明:采用最小噪声分离变换可有效地消除高光谱图像中的噪声;完好无损和表面有疤痕的猕猴桃样本在700~810 nm以及810~1 000 nm的光谱反射率值具有明显的差异,选取785.98 nm处的光谱反射率值为0.30~0.56以及982.59 nm处的光谱反射率值为0.54~0.73作为区分猕猴桃正常区域和表面疤痕区域的阈值条件,进一步利用阈值分割方法对60个完好无损的和60个表面有疤痕的猕猴桃进行识别,正确识别率分别为98.3%和95.0%,说明高光谱成像技术可用于猕猴桃表面疤痕的快速无损识别。     

Hardware-based image processing for high-speed inspection of grains

A high-speed, low-cost, image-based sorting device was developed to detect and separate grains having slight color differences or small defects. The device directly combines a complementary metal–oxide–semiconductor (CMOS) color image sensor with a field-programmable gate array (FPGA) which was programmed to execute image processing in real-time, without the need of an external computer. Spatial resolution of the imaging system is approximately 16 pixels/mm. The system includes three image sensor/FPGA combinations placed around the perimeter of a single-file stream of kernels, so that most of the surface of each kernel is be inspected. A vibratory feeder feeds kernels onto an inclined chute that kernels slide down in a single-file manner. Kernels are imaged immediately after dropping off the end of the chute and are diverted by activating an air valve. The system has a throughput rate of approximately 75 kernels/s per channel which is much higher than previously developed image inspection systems. This throughput rate corresponds to an inspection rate of approximately 8 kg/h of wheat and 40 kg/h of popcorn. The system was initially developed to separate white wheat from red wheat, and to remove popcorn having blue-eye damage, which is indicated by a small blue discoloration in the germ of a popcorn kernel. Testing of the system resulted in accuracies of 88% for red wheat and 91% for white wheat. For popcorn, the system achieved 74% accuracy when removing popcorn with blue-eye damage and 91% accuracy at recognizing good popcorn. The sorter should find uses for removing other defects found in grain, such as insect-damaged grain, scab-damaged wheat, and bunted wheat. Parts for the system cost less than $2000, suggesting that it may be economical to run several systems in parallel to keep up with processing plant rates.     

Hyperspectral waveband selection for internal defect detection of pickling cucumbers and whole pickles

Hyperspectral imaging under transmittance mode has shown potential for detecting internal defect, however, the technique still cannot meet the online speed requirement because of the need to acquire and analyze a large amount of image data. This study was carried out to select important wavebands for further development of an online inspection system to detect internal defect in pickling cucumbers and whole pickles. Hyperspectral transmittance/reflectance images were acquired from normal and defective cucumbers and whole pickles using a prototype hyperspectral reflectance (400-740 nm)/transmittance (740-1000 nm) imaging system. Up to four-waveband subsets were determined by a branch and bound algorithm combined with the k-nearest neighbor classifier. Different waveband binning operations were also compared to determine the bandwidth requirement for each waveband combination. The highest classification accuracies of 94.7 and 82.9% were achieved using the optimal four-waveband sets of 745, 805, 965, and 985 nm at 20 nm spectral resolution for cucumbers and of 745, 765, 885, and 965 nm at 40 nm spectral resolution for whole pickles, respectively. The selected waveband sets will be useful for online quality detection of pickling cucumbers and pickles.     

四川省商品肉猪胴体品质变化的研究

利用FOM肉脂仪分别于2002年10月和2004年10月对四川省内1988头和4553头商品肉猪的胴体品质进行了屠宰线上的随机抽样测定。测量性状为猪左半胴体最后肋骨处背膘厚(P2)和倒数第3、4肋骨间背膘厚(RF)及眼肌厚度(RM),瘦肉率由相关回归公式估测,并随机抽取40头个体测定其部分肉质性状。分析了不同年度、不同胴体等级间各性状的变异程度、相关性及变化趋势,并与国外相关数据进行比较。结果表明:两年来四川省商品肉猪的胴体品质有了明显提高,瘦肉率提高了0.48%,年均提高0.24%,胴体重提高了4.73kg,年均提高2.37kg,且肉质良好。     

基于激光图像的水果表面农药残留检测试验研究

初步探讨了运用激光图像技术来检测水果表面农药残留的方法。应用自行研制的激光图像采集系统获取水果光谱图像,通过对比研究水果表面不喷洒农药、喷洒农药、喷洒不同浓度农药的水果图像特征分析可知,一般未喷洒农药的水果灰度图像灰度值分布在1~100范围内像素点总数低于喷洒农药后的水果灰度图像;喷洒不同浓度农药水果的图像像素灰度值在1~100范围内像素点个数也有明显的差别;但喷洒农药的浓度与像素值聚集在0~100范围内的像素个数之间规律性不明显,有待进一步研究。试验研究表明,用激光图像技术检测水果表面的是否存在农药残留是可行的。     

基于荧光高光谱图像的柑桔糖度无损检测

采用632nm的连续波激光作为激发光,应用激光诱导荧光高光谱成像技术对柑桔的糖度值进行无损测量。先将该激光照射到南丰蜜桔和脐橙样品上,后用高光谱图像采集系统收集诱导出的荧光散射图像;接下来对荧光散射图像进行分析,选取100×50像素的荧光区域作为感兴趣区域(ROIs);再提取感兴趣区域在波长700～1000nm的光谱值作为荧光高光谱图像数据;最后用线性回归方法建立荧光高光谱图像数据预测果实糖度值的预测模型。结果表明,该模型预测柑桔糖度值的相关系数分别为南丰蜜桔的R=0.970,脐橙的R=0.960。因此可以看出,应用激光诱导荧光高光谱成像对柑桔糖度值进行无损检测是一种很有效的方法。     

基于高光谱成像技术的茄子叶片灰霉病早期检测

为建立基于高光谱成像技术的茄子叶片灰霉病早期检测方法,利用高光谱成像系统获取120个茄子叶片在380~1031nm范围的高光谱图像数据,通过主成分分析(PCA)对高光谱数据进行降维,并从中优选出3个特征波段下的特征图像,截取200×150的感兴趣区域图像(ROI),并从每幅特征图像中分别提取均值、方差、同质性、对比度、差异性、熵、二阶矩和相关性等8个基于灰度共生矩阵的纹理特征变量,通过连续投影算法(SPA)提取13个特征变量, 利用最小二乘支持向量机(LS‐SVM)构建茄子叶片灰霉病早期鉴别模型,模型判别准确率为97.5%.说明高光谱成像技术可以用于茄子叶片灰霉病的早期检测.