基于改进Faster R-CNN和Deep Sort的棉铃跟踪计数

棉铃作为棉花重要的产量与品质器官,单株铃数、铃长、铃宽等相关表型性状一直是棉花育种的重要研究内容。为解决由于叶片遮挡导致传统静态图像检测方法无法获取全部棉铃数量的问题,提出了一种以改进Faster R-CNN、Deep Sort和撞线匹配机制为主要算法框架的棉铃跟踪计数方法,以实现在动态视频输入情况下对盆栽棉花棉铃的数量统计。采用基于特征金字塔的Faster R-CNN目标检测网络,融合导向锚框、Soft NMS等网络优化方法,实现对视频中棉铃目标更精确的定位;使用Deep Sort跟踪器通过卡尔曼滤波和深度特征匹配实现前后帧同一目标的相互关联,并为目标进行ID匹配;针对跟踪过程ID跳变问题设计了掩模撞线机制以实现动态旋转视频棉铃数量统计。试验结果表明：改进Faster R-CNN目标检测结果最优,平均测量精度mAP₇₅和F1值分别为0.97和0.96,较改进前分别提高0.02和0.01;改进Faster R-CNN和Deep Sort跟踪结果最优,多目标跟踪精度为0.91,较Tracktor和Sort算法分别提高0.02和0.15;单株铃数计数结果决定系数、均方...

Cotton Boll Tracking and Counting Based on Improved Faster R-CNN and Deep Sort

Cotton boll is an important yield and quality organ of cotton. The research on phenotypic traits such as boll number per plant, boll length and width is of great importance in cotton genetics and breeding research. In order to obtain the accurate number of bolls, a boll tracking and counting method was proposed based on the improved Faster R-CNN and Deep Sort to realize cotton boll measurement based on the rotating video. First of all, a simple video captured device was designed for the cotton plant. And then the feature pyramid network (FPN), Guided Anchoring and Soft NMS methods were adopted to improve the original Faster R-CNN detection network, in which the FPN was used to promote the ability for small targets recognition, Guided Anchoring was applied to generate the Anchors with appropriate size, and the Soft NMS was adopted to mitigate the mistaken deletion of overlapping targets. As a result, the improved Faster R-CNN outperformed the other models, including RetinaNet,SSD, Faster R-CNN, YOLO v5 and YOLOF. The mAP75 and F1 of improved Faster R-CNN was 0.97 and 0.96 respectively, which was 0.02 and 0.01 higher than that of the original Faster R-CNN model. After that, Deep Sort was used to realize the match of the same target in different frames through Kalman filter and deep association metric, and the ID of the same target was matched. In order to solve the ID switch problem, the mask collision mechanism was developed. When the matched cotton boll passed through the mask region from right to left, the ID of the cotton boll would be recorded and the number of the cotton boll would be added, which was proved to significantly reduce the mistaken counting caused by ID switch. Finally, the specialized software was designed based on the improved Faster R-CNN, Deep Sort and mask collision mechanism. The results showed that the tracking result RMOTA was 0.91, which was 0.02 higher than that of Tracktor algorithm, and 0.15 better than that of Sort algorithm, respectively. The measurement results of coefficient of determination, mean square error, mean absolute error and mean absolute percentage error of the bolls number were 0.96, 1.19, 0.81 and 5.92% respectively, which had high consistence with the manual measurement, and it could realize the high precision counting of cotton bolls based on the specialized software. In conclusion, the research demonstrated an effective tool for cotton bolls measurement, which was beneficial to the cotton breeding research.