基于双通路融合网络的用户视频情感识别OA
Emotion recognition in user-generated video based on dual-path fusion network
用户视频的情感识别面临内容动态变化、画面主题多样及情感分布稀疏等挑战.针对此问题,提出一种双通路融合网络DPFNet.DPFNet的核心是基于注意力机制的双通路编码结构,主路径建模视觉嵌入的时序依赖关系,辅助路径则引入情感辅助文本以重建和约束视觉特征以适应多样主题;进一步,通过双通路解码器并行解码两路特征,捕获稀疏情感表达;最后,利用动态门控机制自适应融合双通路解码结果,有效抑制与分类无关的通道干扰.在两个公开数据集上,DPFNet分别取得了60.2%和63.7%的分类准确率.与当前主流模型相比,所提方法在保持相当分类性能的前提下,仅依赖视觉单模态输入即可实现更高效识别,显著降低了模型复杂度与计算开销.
Facing challenges from dynamic content,diverse themes,and sparse emotion distributions in user-generated videos,this paper developed a dual-path fusion network(DPFNet).The core of DPFNet was a dual-path encoding structure with attention mechanisms.The main path modeled temporal dependencies of visual embeddings,while an auxiliary path introduced emotional auxiliary text to reconstruct and constrain visual features for adaptation to diverse themes.Then a dual-path decoder processed both paths in parallel to capture emotion-relevant visual content.Finally,a dynamic gating mechanism adaptively fused the decoded features to suppress task-irrelevant interference.Experiments on two public datasets show that DPFNet achieves classification accuracies of 60.2%and 63.7%,respectively.DPFNet demonstrates more efficient recognition using only visual modality input while maintaining comparable classification performance,significantly reducing model complexity and computational costs.
刘玉杰;董振阳
中国石油大学(华东)青岛软件学院、计算机科学与技术学院山东省智能油气工业软件重点实验室,山东 青岛 266580中国石油大学(华东)青岛软件学院、计算机科学与技术学院山东省智能油气工业软件重点实验室,山东 青岛 266580
信息技术与安全科学
用户视频视频情感识别视觉-语言模型辅助文本嵌入语义查询向量双路解码器动态门控融合
user-generated videovideo emotion recognitionvision-language modelauxiliary text embeddingsemantic querying vectordual-path decoderdynamic gating fusion
《计算机应用研究》 2026 (8)
2278-2285,8
国家重点研发计划资助项目(2019YFF03018000)国家自然科学基金资助项目(61379106)山东省自然科学基金资助项目(ZR2013FM036,ZR2015FM011)
评论