结合生成对抗网络和跨模态映射融合的多模态情感分析OA

In the field of smart education,multi-modal sentiment analysis technology deeply analyzes classroom audio and video data to accurately assess students'emotional states,providing a basis for optimizing teaching.However,existing methods face limitations,including insufficient feature extraction from single modality,which leads to information loss,and unstable multi-modal data fusion,resulting in suboptimal fusion representations.To address these issues,this paper proposes a multi-modal sentiment analysis method that combines adversarial networks with cross-modal mapping fusion.First,a Generative Adversarial Network(GAN)is employed to enhance the text and audio features extracted by BERT and LSTM,respectively,while DeiT is used to extract visual features,ensuring the sufficiency of features from each modality.Second,a cross-modal mapping fusion method is introduced,combining an improved cross-modal Transformer and a unique fusion gating mechanism to facilitate inter-modal information interaction and capture emotion-related information.Additionally,a Unimodal Label Generation Module(ULGM)is incorporated to help the model learn the unique characteristics of each modality.Experiments conducted on the MOSI and the MOSEI datasets show that the proposed method significantly improves sentiment analysis accuracy,thereby validating the effectiveness of the proposed model.