基于单摄像机视频的鱼类三维自动跟踪方法初探

为提高鱼类行为学数据的提取效率, 实验提出了一种基于单摄像机的鱼类三维观测方法, 将防水镜面安装在实验用鱼缸上方, 模拟一台由上向下拍摄的摄像机, 实现了单摄像机三维成像。同时运用多目标跟踪的IMMJPDA(interacting multiple model joint probabilistic data association)算法, 实现了鱼类运动的三维实时自动跟踪, 并通过摄像机倾斜矫正和摄像机标定提高了测量精度。通过对6条红鼻剪刀鱼的跟踪, 实验结果显示: 本方法可正确区分、提取和跟踪鱼群个体以及它们的镜像, 自动输出鱼的三维坐标、实时速度、方向等参数, 并生成完整的鱼类行为三维轨迹图。

Preliminary studies on an automated 3D fish tracking method based on a single video camera

The study of fish behavior lays an important foundation for comprehending of fish migration routes, improving fishing efficiency and the protection of fishery resources. A large number of data are necessary in the study, such as stress response, cluster, migration and other measured data. However, getting these data is a time-consuming process. As fish behavior is often recorded in the form of video and a stereo camera recording system is popularly used for measurement and observation in the laboratory study, how to extract the data of fish behavior efficiently from the video has been a major problem in the study of fish behavior. By far fish 3D coordinate is usually calculated by hand, or by self made software which turns an importing fish 2D coordinate into a 3D one. In order to improve fish behavior data extraction efficiency, this paper presents an automated 3D fish tracking method based on a single video camera. A waterproof mirror was set above the experimental aquaria to simulate a camera shooting from the top, which provided a way to use a single camera for 3D imaging. We extract the data of fish behavior automatically by 3D fish tracking method which is divided into four parts: distortion calibration of single camera system, transfer formula between image coordinate to world coordinate, the automated tracking algorithm of fish movement and the automated output of fish behavior 2D and 3D data. Tests find out that while the distance between the camera and the aquaria is 1.5 m, the distortion calibration result shows the pixel error is much more acceptable which is about 0.1 pixels. As the camera tilted slightly during the experiment, the shape of the aquaria in the images changed. So based on the processing of Free-Form Deformation, the deformation of images is rectified during coordinate transform process. Then we implemented the algorithm of Interacting Multiple Model Joint Probabilistic Data Association (IMMJPDA) to automatically track fishes in 3D and output fish behavior data. The result of 6 Hemigrammus rhodostomus tracking experiment shows that: IMMJPDA algorithm can deal with the key issues during fish tracking system, which enables the method to extract individual fish from video images, construct their tracks, output 3D positions and speeds, and finally generate a complete 3D movement track drawing for fish behavior analysis. In a dense clutter situation JPDA requires a fairly large amount of computation to evaluate the joint probabilities. We combined Nearest Neighbor algorithm and JPDA algorithm to reduce the computational burden.