首页|期刊导航|Computers, Materials & Continua|MDGET-MER:Multi-Level Dynamic Gating and Emotion Transfer for Multi-Modal Emotion Recognition

MDGET-MER:Multi-Level Dynamic Gating and Emotion Transfer for Multi-Modal Emotion RecognitionOA

中文摘要

In multi-modal emotion recognition,excessive reliance on historical context often impedes the detection of emotional shifts,while modality heterogeneity and unimodal noise limit recognition performance.Existing methods struggle to dynamically adjust cross-modal complementary strength to optimize fusion quality and lack effective mechanisms to model the dynamic evolution of emotions.To address these issues,we propose a multi-level dynamic gating and emotion transfer framework for multi-modal emotion recognition.A dynamic gating mechanism is applied across unimodal encoding,cross-modal alignment,and emotion transfer modeling,substantially improving noise robustness and feature alignment.First,we construct a unimodal encoder based on gated recurrent units and feature-selection gating to suppress intra-modal noise and enhance contextual representation.Second,we design a gated-attention crossmodal encoder that dynamically calibrates the complementary contributions of visual and audio modalities to the dominant textual features and eliminates redundant information.Finally,we introduce a gated enhanced emotion transfer module that explicitly models the temporal dependence of emotional evolution in dialogues via transfer gating and optimizes continuity modeling with a comparative learning loss.Experimental results demonstrate that the proposed method outperforms state-of-the-art models on the public MELD and IEMOCAP datasets.

Musheng Chen;Qiang Wen;Xiaohong Qiu;Junhua Wu;Wenqing Fu

School of Software Engineering,Jiangxi University of Science and Technology,Nanchang,330013,China Nanchang Key Laboratory of Virtual Digital Engineering and Cultural Communication,Nanchang,330013,ChinaSchool of Software Engineering,Jiangxi University of Science and Technology,Nanchang,330013,ChinaSchool of Software Engineering,Jiangxi University of Science and Technology,Nanchang,330013,China Nanchang Key Laboratory of Virtual Digital Engineering and Cultural Communication,Nanchang,330013,ChinaSchool of Software Engineering,Jiangxi University of Science and Technology,Nanchang,330013,ChinaSchool of Software Engineering,Jiangxi University of Science and Technology,Nanchang,330013,China

信息技术与安全科学

Multi-modal emotion recognitiondynamic gatingemotion transfer modulecross-modal dynamic alignmentnoise robustness

《Computers, Materials & Continua》 2026 (3)

P.872-893,22

funded by“the Fanying Special Program of the National Natural Science Foundation of China,grant number 62341307”“the Scientific research project of Jiangxi Provincial Department of Education,grant number GJJ200839”“theDoctoral startup fund of JiangxiUniversity of Technology,grant number 205200100402”.

10.32604/cmc.2025.071207

评论