一种基于证据驱动的多模态对话混合情绪状态转移模型OA

Multimodal conversational emotion recognition aims to predict the emotional state of each utterance in a dialogue by fusing information from multiple modalities,such as text,audio,and video.In this task,emotional states typically exhibit dy-namic evolution characteristics,while information inconsistencies across modalities introduce significant interference,which jointly increases the complexity of emotion recognition.To address the above issues,this study proposed an evidence-driven hybrid emotion state transition model.This model fused textual,acoustic and visual modalities to capture feature variations be-tween adjacent utterances,and fused intra-utterance local features with inter-utterance dynamic changes to construct evidence representations that reflected the intensity of emotional variation.Based on the obtained evidence representations,and com-bined with modality conflicts and speaker differences,the evidence representations guided a hybrid emotion state transition mechanism to adaptively regulate the two evolution patterns of continuity and abruptness.The model further combined the de-rived emotional states with dialogue context to complete utterance-level emotion prediction.Experimental results show that the proposed model achieves accuracies of 74.12％and 67.32％,and F1 scores of 74.03％and 66.11％on IEMOCAP and MELD,the benchmark datasets for multimodal conversational emotion recognition,which represents a significant improvement over existing baseline models.