基于EM-PCNN的果园苹果异源图像配准方法

针对果园环境下双目采集系统采集的飞行时间（Time of Flight，ToF）与可见光异源图像间匹配精度差的问题，该研究提出一种基于局部峰值的目标显著区域提取策略及最大期望算法的脉冲耦合神经网络分割的ToF与可见光果园苹果图像配准方法。首先，利用高斯差函数计算可见光图像中显著性区域，对可见光图像的红绿分量进行预处理；然后，以图像局部灰度值的二维正态分布作为目标分量，使用Otsu提取具有固定阈值的前景作为局部峰值提取策略，对ToF与可见光图像初步筛选特征区域，利用最大期望算法改进脉冲耦合神经网络（Pulse Coupled Neural Network，PCNN）动态阈值，使用局部图像梯度计算链接强度计算链接强度，利用图像区域方差改进终止条件，提出一种基于最大期望的脉冲耦合神经网络（Pulse Coupled Neural Network based on Expectation Maximization，EM-PCNN）算法对预选区域进行精细化分割；接着计算连通区域不变矩，利用不变矩特征原理寻找目标中心同名点，进一步筛选特征区域；最后，同名点进行随机抽样一致算法（Random Sample Consensus，RANSAC）提纯，将提纯后的同名点坐标代入变换模型计算模型参数，完成配准。在不同光照条件下均方根误差达3.05～4.75，配准点达3～5。EM-PCNN算法对两组ToF置信图像分割的准确率分别为96.62%和73.84%。试验结果表明该方法对双目采集系统采集的ToF与可见光异源果园苹果图像可实现较好配准效果，且对图像平移、旋转、缩放均具有可抗性。研究结果对ToF与可见光异源图像在果园环境下自动配准提供了技术参考。

Heterologous sources images in the apple orchard registration method using EM-PCNN

Image registration has been a key preprocessing technique for heterogeneous images. Two kinds of image data were transformed into the same coordinate system by matching, combining spatial information and semantic information. Multi-source images can be gained to reduce the limitations of a single data source. This study aims to improve the matching accuracy between Time of Flight (ToF) and visible heterologous images collected by the binocular acquisition system in an orchard environment. A target salient region extraction was proposed using the local peak, pulse coupled neural network (pCNN) segmentation and the expectation-maximization for the ToF and visible image registration in an apple orchard. Firstly, the significant region in the visible image was calculated to preprocess the red and green components of the visible image, according to the Gaussian difference function. Taking the two-dimensional normal distribution of the local gray value in the image as the target component, the maximum interclass variance Otsu was used to extract the prospect with a fixed threshold as the local peak extraction strategy, then to preliminarily screen the characteristic areas of ToF and visible images, in order to improve the pCNN dynamic threshold using the maximum expectation. The link strength was calculated using the local image gradient. An Expectation-Maximization (EM) pCNN was proposed to refine the pre-selected region using the image region variance for a better termination condition. The invariant moment of the connected region was calculated to locate the same name point of the target center using the invariant principle, where the characteristic region was further screened. Finally, the same name points were purified by the Random Sample Consensus (RANSAC), where the purified coordinates of the same name points were substituted into the transformation model to calculate the model parameters for the registration. The experimental images were collected under three conditions, including a sunny day, shade, and weak light, in order to simulate different lighting conditions. The translation, spatial rotation, and scaling were performed on each group of ToF and visible images. The experimental results show that the EM-PCNN was better than the traditional segmentation for the image with less obvious color difference between fruit and growth background. Under normal conditions, the segmentation accuracy and the exposure were 96.62% and 73.84% lower, respectively. After that, the feature regions were screened to perform fine segmentation using EM-PCNN. There were smaller differences in regional features between the ToF and visible light, compared with the traditional Maximally Stable Extremal Regions (MSER), indicating a more accurate range. Harris algorithm was suitable for the image registration with the small scale change and rotation angle. The scale-invariant feature transform (Sift) algorithm presented the resistance to the translation, rotation, and scaling, where the complete registration was realized with the rotation at the spatial level. Consequently, the local peak extraction and EM-PCNN segmentation can be widely expected to register the ToF and visible images in the apple orchard, where the root mean square error was 3.05-4.75 under different lighting conditions, and the homonymous point was 3-5. The excellent registration was achieved for the ToF and visible heterologous images in an apple orchard under the binocular acquisition system, indicating better resistance to the image translation, rotation, and scaling.