模态缺失下基于提示学习的多模态情感分析OA

To address the issues in multimodal sentiment analysis under missing modalities,such as the complex fine-tuning of Transformer models,the indiscriminate use of completed modality data,and the weakened information complementarity caused by modality missing,this paper proposed a prompt-learning-based multimodal sentiment analysis model under missing modali-ties.The model firstly introduced modality-missing prompts to guide the model in identifying whether the current input modality was missing.Secondly,it constructed a quality evaluation mechanism to suppress the interference of low-quality completed mo-dality data on the model.On this basis,it designed and added modality-missing combination prompts into the Transformer of the backbone network to guide the model to dynamically adjust attention computation and cross-modal interaction,improving the model's adaptability to missing-modality scenarios,while avoiding complex fine-tuning of the Transformer backbone and reducing computational cost.Finally,it added a shared fusion layer to map the features of each modality into a unified shared representation space,learning shared semantic information and enhancing cross-modal semantic consistency and information complementarity.Experimental results show that,under six missing-modality combinations,the model improve the average Acc-2 and F1 scores on the CMU-MOSI,IEMOCAP,and CH-SIMS datasets by 1.04～1.87 percentage points compared with the second-best model.In addition,the trainable parameters account for only 6.3％of the total parameters,verifying the ef-fectiveness,robustness,and parameter efficiency of the proposed model.