多尺度分解双寻优策略SPCNN的果园苹果异源图像融合模型

针对单一的自然场景图像信息不能满足准确识别果实和精准定位目标的要求，提出一种多尺度分解双寻优策略简化脉冲耦合神经网络（Simplified Pulse Coupled Neural Network, SPCNN）的飞行时间（Time of Flight，ToF）与可见光果园苹果图像融合模型。对SPCNN模型引入带参数优化的双寻优策略，对非下采样轮廓波变换（Nonsubsampled Contourlet Transform, NSCT）融合规则进行改进。模型包括配准模块、编码区、多尺度分解模块、单目标SPCNN融合模型、多目标SPCNN融合模型、解码区。模型改进了SPCNN模型的参数优化方式以及迭代次数，模型自适应点火次数较低，在3～7次左右，具有点火次数低、自适应分割、效率高的优点。中光15:00时段点火识别成功率达到了100.00%，点火分割时间达到最低91.91s。与其他融合模型比较，模型在强光12:00、中光15:00、弱光18:20、19:00时段融合图像识别成功率达到100.00%；融合时间低于SPCNN模型，达到最低92.68s。模型识别精度最优达到了100.00%、融合耗时最低达到了92.68s、模型大小较SPCNN低一个数量级，可补充和完善图像层次融合理论和方法。

Heterologous image fusion with multi-scale decomposition and dual optimization SPCNN in an apple orchard

Heterologous image fusion has been widely used to integrate multiple images into one. The fused images also present a higher definition, more significant edge intensity, and more information than the source image. There are different characteristics of image data collected by the various types of sensors. Among them, the depth sensor imaging used the Time of Flight (ToF) to realize the distance calculation using the ToF near-infrared light. The beneficial supplement has been commonly used for the visible light camera. The broad application can also be expected in the agriculture, medical treatment, quality inspection, and vision fields. However, the image acquisition of a single natural scene cannot fully meet the requirements for rapid and accurate identification of the fruits and positioning targets. The image fusion can be extended to the heterologous vision system using multi-objective optimization, particularly in the field of natural scenes. In this study, a multi-scale decomposition and dual optimization strategy was proposed to simplify the ToF and visible-light image fusion in an apple orchard using the simplified pulse coupled neural network (SPCNN). A double strategy with parameter optimization was introduced into the SPCNN model for the fusion of nonsubsampled contourlet transform (NSCT). The model included the registration module, coding area, multi-scale decomposition module, single target SPCNN fusion model, multi-target SPCNN fusion model, and decoding area. The heterologous vision system was also used to accurately register the ToF and visible light images. Four parameters of SPCNN model were encoded, including the link channel feedback term, link strength, dynamic threshold attenuation factor, and dynamic threshold amplification factor. The NSCT was used to decompose the image at multiple scales. The fusion rules in the SPCNN model were adopted with the improved artificial bee colony algorithm and double optimization, including the single- and multi-objective parameter optimization. Each binary vector was converted into the real parameters using the decoding area. The objective function of the double optimization was used as the iteration termination of the SPCNN model. Finally, the heterogeneous image fusion was implemented after the multi-scale inverse transformation. There was improved parameter optimization and iteration times of SPCNN model. The adaptive ignition times of the model were relatively low (about 3-7 times), indicating low ignition times, adaptive segmentation, and high efficiency. The success rate of ignition recognition reached 100.00%, and the minimum duration of ignition division reached 91.91s at 15:00. Specifically, the success rate of fusion image recognition also reached 100.00% under different periods, including strong, medium, and weak light at 12:00, 15:00, 18:20, and 19:00, compared with the rest fusion models. The fusion time was much lower than that of SPCNN model, with a minimum of 92.68s. The four fusion indexes were the largest in the weak light period of 18:20, including the average gradient, correlation coefficient, mutual information, entropy, and spatial frequency. The proposed model presented an excellent performance in accuracy, time-consuming, and model size. The finding can provide a supplement to the image hierarchical fusion.