面向智能座舱的多源混合模态数据集及层次化融合分类方法OA

The scarcity of open-source data for intelligent cockpits in the driving domain is characterized by limited modality dimensions,insufficient annotations,and restricted scene diversity.To address these challenges,a multi-source hybrid-modality dataset has been constructed.This dataset incorporates RGB,depth,and infrared visual data,along with structured textual data detailing vehicle information and driving scenarios.A dual-layer annotation scheme is applied to capture ten behavior categories.Leveraging this dataset,a hierarchical multi-modal fusion framework is proposed to en-hance feature extraction via cross-modal information exchange and semantically guided fusion mechanisms.Experi-ments on video classification tasks reveal significant improvements in environmental understanding when combining RGB data with additional modalities.Using the full range of modalities leads to a 15.75％increase in accuracy com-pared to using only RGB data.These results validate the effectiveness of the multi-source hybrid-modality dataset in ad-vancing intelligent cockpit systems.