基于视觉-语言预训练模型的高光谱-激光雷达联合分类方法OA

To address the challenge of inaccurate land-cover classification caused by differences in spatial resolution,data heterogeneity,and limited labeled samples in multimodal remote sensing data,we investigate the joint classifica-tion of Hyperspectral Imagery(HSI)and LiDAR data.We propose a Semantic-aware Cross-modal Fusion Network(SCF-Net).First,a lightweight patch encoder transforms the input data into RGB-compatible feature maps,which are then fed into a Contrastive Language-Image Pre-training(CLIP)-based visual encoder enhanced with learnable prompts.To efficiently integrate multimodal information,an adaptive cross-modal fusion architecture is employed,fea-turing grouped linear projection and a relation-aware interaction module that enables dynamic spatial feature exchange at low computational cost.For semantic discrimination,attribute-category textual prompts are generated,and classifi-cation is performed by computing the cosine similarity between visual and textual embeddings,followed by a TopK at-tribute averaging strategy.Experiments on the Houston 2013,MUUFL,and Trento datasets demonstrate that SCF-Net outperforms eight state-of-the-art fusion methods,achieving improvements of over 2.88%in overall accuracy,2.69%in average accuracy,and 3.02%in Kappa coefficient,while maintaining high parameter efficiency.Ablation studies further validate the effectiveness of each component.This network offers a novel paradigm for integrating multi-modal remote sensing data with large-scale vision-language pre-trained models in complex classification tasks.