文本导向的多任务多模态情感感知分析模型OA

To address the limitations of existing multimodal sentiment analysis models in underemphasizing the text modality and utilizing context,a multi-task multimodal emotion perception analysis model that integraces text augmentation and cross-modal interactive attention mechanisms is proposed.Specifically,we introduce a GPT-based text augmentation technique to enhance the recognition of emotional words and reinforce the text modality.A cross-modal interactive attention mechanism is then employed to fuse visual,audio and textual information effectively.Moreover,a homoscedastic uncertainty loss function is applied to optimize weight adjustments across multiple tasks.On the CMU-MOSI and CMU-MOSEI datasets,the proposed model achieves ACC2 and F1 scores of 87.2%and 85.8%respectively,outperforming the baseline model.This demonstrates the effectiveness of the proposed model and its significant improvement in multimodel perception analysis performance.