相机与毫米波雷达融合检测农机前方田埂

为满足自主作业农机地头转弯的需求，解决单传感器检测获取信息不足的问题，该研究提出了将相机与毫米波雷达所获取数据融合的多传感前方田埂检测方案，利用视觉检测得到田埂形状后辅助滤除毫米波雷达中干扰点进而得到田埂的距离和高度信息。在视觉检测方面，根据前方田埂在图像中的分布特点，提出了基于渐变重采样选取部分点的加速处理方式，并在此基础上利用基于11维颜色纹理特征的支持向量机进行图像分割和基于等宽假设的几何模型特征进行误分类点剔除，然后拟合提取图像中田埂边界。在毫米波雷达检测方面，提出了竖直放置毫米波雷达的检测方式，以克服安装高度与地形颠簸的影响，并获得前方田埂的高度信息。将相机与毫米波雷达获取的数据进行时空对齐后，利用视觉检测结果滤除毫米波雷达干扰点，并将毫米波雷达获得的单点距离信息进行扩展，形成维度上的数据互补，获得前方田埂的形状、距离、高度等更加丰富准确的信息。测试结果表明，在Nvidia Jetson TX2主控制器上，基于视觉的检测平均用时40.83 ms，准确率95.67%，平均角度偏差0.67°，平均偏移量检测偏差2.69%；基于融合算法的检测平均距离检测偏差0.11 m，距离检测标准差6.93 cm，平均高度检测偏差0.13 m，高度检测标准差0.19 m，可以满足自主作业农机的实时性与准确性要求。

Detection of ridge in front of vehicle based on fusion of camera and millimeter wave radar

Detection of the front ridge is an important step for navigation and path planning of autonomous agricultural vehicles. In single sensor detection, it is hard to acquire enough information, such as shape, distance, and height, because of the complex environment in the field. In this study, the novel detection of the front ridge was proposed to integrate the camera and millimetre-wave radar. The camera was used to collect the shape, while the radar was used to collect the distance and height of the front ridge. More detailed information of the front ridge was achieved after fusion of the data acquired by the camera and millimetre-wave radar. In visual detection, the distribution characteristics of the front ridge were used in the images, while the gradient resampling was used to accelerate image processing. Only less than 1% of the total needed to be processed. A support vector machine (SVM) was then applied with 11-dimensional colour-texture features in image segmentation. The 11-dimensional colour-texture features contained three-dimensional colour features in RGB colour space, four-dimensional colour features in HSI colour space, and four-dimensional texture features in gray level co-occurrence matrix, indicating both colour and texture features of the front ridge. Furthermore, the equal-width hypothesis of the geometric feature was used to obtain a more accurate shape of the front ridge. The equal-width hypothesis referred to that there were no sharp curvature changes of the front ridge in the images. Some misjudgement points were filtered in this hypothesis. Millimetre-wave radar was installed vertically in the millimetre-wave radar detection. Compared with the common horizontal one, the vertical installation was used to ensure the installation height and bumpy ground, while the height of ridge at the same time. In fusion detection, the millimetre-wave radar data was transferred to the image coordinate system via coordinate transformation formula and pinhole imaging model. The visual detection was then used to filter interference points in the millimetre-wave radar data. An accurate distance of the front ridge was captured, and the interference points in the millimetre-wave radar data were filtered easily using the coupled camera and millimetre-wave radar. Both camera and millimetre-wave radar were two-dimensional sensors, but after fusion, the three-dimensional information was achieved, like shape, distance, and height of the front ridge. The camera and millimetre-wave radar enhanced each other. The radar was placed at different heights and angles in both horizontal and vertical installation, in order to verify the vertical placement of radar. Tests showed that the horizontal installation was greatly affected by the installation height and terrain turbulence, but the vertical installation effectively overcame these effects. A dataset was recorded to verify the correctness of fusion, including 300 images and 50 groups of fusion data with different distance and shooting angles of the front ridge. The test was performed on the Nvidia Jetson TX2 hardware platform, where the visual detection spent 40.83 ms per image, and the accuracy was 95.67%, the average angle deviation was 0.67°, the average offset deviation was 2.69e-2. The accuracy was slightly reduced by 1.33%, the average angle deviation was slightly reduced by 0.04°, the average offset deviation was slightly reduced by 0.14e-2, but the detection speed was improved by 794.11 ms, compared with the traditional whole image processing. The average deviation of distance detection was 0.11 meters in fusion detection, the standard deviation of distance detection was 6.93e-2 m, and the average deviation of height detection was 0.13 m. Consequently, the standard deviation of height detection was 0.19 m. The fusion detection of the camera and millimetre-wave radar can meet the requirements of real-time and accuracy for autonomous agricultural vehicles.