基于文本层次化信息增强的多模态情感分析OA

Multimodal sentiment analysis aims to recognize human emotions from textual,visual and audio information.Previous approaches have not fully considered the use of textual features to enhance the emotional information present in the visual and audio modalities,as well as the correlation between the different modalities.To address these issues,this paper proposes a multimodal sentiment analysis model based on text hierarchical information enhancement(THIE).The model first uses Transformer to extract textual,visual and audio modal features in order to get important semantic information.Then,it designs a text hierarchical information enhancement module,which utilizes textual modalities at different levels to fuse relevant emotional information in visual and audio modalities,respectively,so as to achieve the effect of emotional enhancement.Finally,it proposes a multimodal fusion module based on cross-modal attention and gating mechanism,which is used to enhance the correlation existing between different feature information,increase the weight of important features,and suppress relatively unimportant features,thus reducing redundant information.Experiments on the public datasets CMU-MOSI,CMU-MOSEI and CH-SIMS show that the model has improved performance compared with main-stream methods.