CIMOT3D:基于中文引导的单目视角下三维多目标跟踪研究OA

Natural language-driven object tracking parses human-like language descriptions and fuses them with visu-al information to achieve accurate recognition and continuous tracking of specific targets in complex environments.Howev-er,existing methods focus on 2D tracking or 3D single-target tracking,and they have not been effectively extended to 3D multi-target tracking.They lack the capability to align text with multiple candidate targets in 3D visual space and to estab-lish associations.In addition,existing natural language-driven 3D object tracking tasks suffer from redundancy in language descriptions,which makes it hard to track multiple specific targets using flexible and concise instructions as humans do.To address these challenges,this paper introduces a new task,chinese-instruction-based monocular 3D multi-object tracking(CIMOT3D).The paper also constructs a new dataset,CIMOT3D-5k,which contains 5 562 video sequences with human-like Chinese descriptions.Furthermore,this paper designs a neural network model chinese-instruction-based monocular 3D multi-object tracking synchronization tracker(CIMOT3D-SyncTracker)for this task,which consists of a multimodal feature extractor,a vision-language encoder-decoder,and a detection-tracking module.Compared with baseline methods,the pro-posed approach achieves an improvement of 4.1%in tracking accuracy and 5.0%in identity consistency metric on the CIMOT3D-5k dataset,verifying its performance advantage.This paper advances research on vision-language fusion in 3D multi-object tracking and offers new ideas for further exploration in related fields.