苹果采摘机器人中的图像配准技术

为了减少自然环境下的光线干扰，采用一个彩色相机和一个深度相机获取目标物的图像，利用多源传感器信息融合与互补方法，实现多目标图像的精确配准。基于TOF（time of flight）技术的PMD（polarization mode dispersion）相机，能实时获得强度图像和深度信息。以苹果树为研究对象，采用Harris检测提取特征点，在归一化互相关系数法的基础上运用邻域的支持强度实现了PMD图像与彩色图像的同名点配准。对自然场景中共50组图片进行试验验证，该方法顺光条件下正确匹配率达到85.75%，逆光条件下的匹配率是79.57%，能满足光线变化的图像精确配准的要求。

Application of image registration technology in apple harvest robot

Abstract: To reduce the effect of natural light, this paper provides a novel apple harvest robot vision system, which integrates a new technique that combines a color-camera system with a PMD-camera. A registration method of color-camera and the PMD-camera is presented to find precise corresponding color pixel information with range distance data from the PMD-camera. The registration algorithm used in the article has the following steps: feature extraction, feature matching, coordinates transformation and interpolation, and feature extraction and feature matching are the key technologies of all. Firstly, as large portions of corners in multi-source images have high correlation, Harris corner detection based on differential operation and autocorrelation matrix was chosen as the method of extracting image features. Secondly, the Normalized Correlation Coefficient (NCC) algorithm was used to realize many-many matching relationships between color images and PMD images, which is a pre-alignment stage. NCC relies on gray information around the corner and has good ability to noise. Thirdly, combined with the information of corners around corresponding points, the refinement stage is completed by way of calculating the support strength of its neighbor points. Finally, the article gets the final registration image after affine transformation and bilinear interpolation. Fifty groups of apple tree pictures that were taken in a natural scene are used to verify the algorithm, including 28 groups of pictures in front lighting and 22 groups of pictures in backlighting. What's more, statistical results of contrast test with different algorithm used in previous articles is obtained, in which each dataset shows minimum, mean, and standard deviation values of matching rate. The experimental result shows that, the algorithm used in this paper is obvious better than previous articles. The matching rate reaches85.75% in front lighting condition and 79.57% in backlighting condition, which can meet the requirements for accurate image registration. As the matching rate doesn't perform as well as expected, error sources is briefly analyzed. A region of interest (ROI) with a PVC pipe frame is suggested to simplify image registration. The registration algorithm lays a foundation for the later work of multi-source image fusion. Moreover, visual perception or feature extraction can be improved by the combination of two images from different cameras of a scene.