Automatic Radar-Camera Calibration and Fusion for Traffic PerceptionOA

In intelligent transportation systems(ITS),millimeter-wave(MMW)radar-camera fusion has emerged as a cost-effective and viable solution due to low sensor prices.However,deploying such a fusion system in practical far-range scenes faces significant challenges in both sensor calibration and fusion processes.To address these challenges,this paper presents a systematic study from theoretical analysis to practical system deployment.First,we review the status quo of radar-camera fusion systems,comparing existing calibration and fusion paradigms carefully.Through comparative analysis,we find that though feature-level fusion is popular in related research,the target-level fusion is more practical for roadside applications because it is computationally efficient and more robust to depth ambiguity.Second,we introduce an automatic radar-camera calibration and fusion system for real-world traffic perception.This system implements a trajectory-based calibration scheme for spatio-temporal synchronization,specifically tackling the difficulty of identifying distinguishable calibration targets in far-range environments.After calibration,this system applies a robust two-stage target-level fusion method to achieve effective radar-camera fusion in traffic scenes.Finally,we introduce the promising advancements of the proposed system and discuss several open challenges for large-scale and high-safety commercialization.We believe physics-aware self-supervised learning,cooperative perception across roadside devices,and end-to-end perception foundation models are important for future traffic perception systems.