基于视觉显著性和脉冲耦合神经网络的成熟桑葚图像分割

为了提高在自然采摘环境中成熟桑葚机器视觉识别的有效性和鲁棒性,克服图像目标形态小、分布杂散、背景干扰多和光照不均匀等困难,该文提出了一种采用视觉显著性和脉冲耦合神经网络(pulse coupled neural network,PCNN)模型的成熟桑葚图像分割方法。该方法首先将采集的图像映射到Lab颜色空间,利用空间颜色分量的算术平均值和高斯滤波值之间的差异,构建起桑葚图像的频率调谐视觉显著图;其次,提取采集图像在HSI颜色空间的色调分量,经过均衡化处理后,与视觉显著图进行融合,实现桑葚目标的融合特征表达;最后,通过改进的分层阈值化脉冲耦合神经网络模型进行目标分割以及形态学处理,得到成熟桑葚的识别结果。利用从重庆市天府镇果桑生态园采集到的200余幅桑树挂果图像进行试验,结果表明,该方法能够在不同光照条件的复杂背景下,有效分割出成熟果实,平均误分率为1.87%,优于结合频率调谐视觉显著性的OTSU法(17.73%)、K-means聚类算法(10.69%)、基于Itti视觉显著性的PCNN分割方法(7.34%)和基于GBVS(graph-based visual saliency,GBVS)视觉显著性的PCNN分割方法(5.83%)。研究结果为成熟桑葚果实的智能化识别提供参考。

Image segmentation of ripe mulberries based on visual saliency and pulse coupled neural network

Abstract: In the planting environment, it is difficult to detect the ripe mulberry targets automatically by machine vision because of the small shape, irregular distribution of fruits, non-uniformly illuminating condition and complex background interference. This paper presents a visual segmentation approach of ripe mulberries based on frequency-tuned (FT) saliency map and the pulse coupled neural network (PCNN) model in order to improve the segmentation precision and robustness for ripe mulberries. Firstly, the captured mulberry image is transformed into Lab color space. This color space is designed by the perception of the human eye in the natural color. Consequently, the color feature of the target can be expressed by 3 independent components i.e. L, a, and b. Next, the FT saliency map of mulberries can be computed by a multiscale edge detector. And it typically distinguishes differences between the region of ripe mulberries and the complex background in Lab color space. This saliency is illustrated as the difference matrix which includes the Euclidean distance of each pixel between the Lab vectors in the Gaussian filtered image and the average Lab vectors in the original image. Thereafter, the hue feature is extracted from the HSI (hue, saturation, intensity) color space of the captured mulberry image, and then equalized by contrast limited adaptive histogram equalization (CLAHE), which can enhance the color contrast of the mulberry image without being affected by illumination changes. And the feature image of mulberries is generated by fusing the FT saliency and the hue feature, as an input of the improved multilevel threshold PCNN model. Furthermore, the region of ripe mulberries can be detected by this PCNN model, which is a bio-inspired neural network and derived from synchronous dynamics of neuronal activity in a mammal visual cortex. This model is able to cause the adjacent neurons with similar inputs to pulse synchronously, and be appropriate to segment the small target from the complex background. Finally, the segmentation result of the ripe mulberry is acquired by morphology repair operation, and it ensures the integrity of the target region and the independent noise removal. In order to verify the effect of the method proposed in this paper, there were 200 test images captured from the mulberry ecological park of the Tianfu Town, Chongqing City, China in May, 2016. The acquisition time was often on sunny day and cloudy day, aiming at obtaining images under different lighting conditions, such as balanced illumination, imbalanced illumination and backlight. These images were collected by the Canon EOS70D digital camera with 3 648×5 472 pixels, and zoomed into 720×1 280 pixels to apply to the remote intelligent monitoring system of mulberry growth. The algorithm programming development platform is Visual C++ 2015. The experimental results point out that the average misclassification error (ME) by our method is only 1.87%, infinitely superior to the OTSU method with FT saliency (17.73%), the K-means method (10.69%), the improved multilevel threshold PCNN method based on Itti saliency map (7.34%), and the PCNN method based on graph-based visual saliency (GBVS) image (5.83%). The average segmentation time of a test image based on our method is 2.562 3 s. Consequently, our approach is effective to segment the ripe mulberries in the complicated background and different lighting conditions.