多模态蒸馏驱动的单RGB鲁棒目标检测方法OA

To address the insufficient robustness of lightweight single-RGB object detection models under complex illumination,low visibility,or partial occlusion conditions,as well as the challenges of high hardware cost,complex calibration,and limited power consumption faced by multimodal sensor solutions in edge deployment,this paper proposes multi-modal distillation-driven robust object detection method with single RGB.The method constructs a heterogeneous teacher-student distillation framework in which a multimodal teacher network guides the training phase while only the single-RGB student model is retained for inference.During training,the teacher network is built with RGB images and auxiliary modal images,transferring structural information from thermal infrared,depth,and other auxiliary modalities to a lightweight student network that receives only RGB inputs through degradation modeling,cross-modal semantic projection,cross-modal consistency constraints,and modal-compensation attention mechanisms.During inference,only the single-RGB student network is retained to enable low-cost deployment.To enhance experimental verifiability,comparative experiments,missing-modality generalization experiments,ablation studies,and deployment experiments are conducted on the self-built multimodal training dataset USAB-180K and the public benchmark KAIST dataset.Experimental results demonstrate that the proposed method achieves a 3.4 percentage point improvement in Macro-F 1 over the single-RGB baseline model on the KAIST test set,reduces the false positive rate by 5.6 percentage points,and increases mAP@50 from 62.4％to 72.1％.Under 1080p,30 f/s video stream input,when deployed on the RV1106 industrial vision platform,the model achieves an average end-to-end latency of 31.8 ms and an inference power consumption of 206 mW.The proposed method provides a feasible approach for enhancing the stability of object detection performance on resource-constrained edge devices in complex environments.