基于动态-分层-对抗协同优化的知识增强BERT文本分类模型OA
Classification:A triple enhancement framework combining dynamic,hierarchical,and adversarial mechanisms
像BERT这样的预训练语言模型擅长捕捉通用语言模式,但由于缺乏结构化知识,在特定领域的文本分类中表现不佳.为了使模型在特定领域拥有出色的推理能力,知识注入逐渐成为主流.然而,过度的知识融合会改变句子的正确含义,导致知识噪声(Knowledge Noise,KN)问题.为了克服领域特定知识缺口和知识噪声在文本分类领域的影响,本文提出了一个三重增强的BERT框架,具体包括:1)知识增强动态注意力(Knowledge-Enhanced Dy-namic Attention,KEDA);2)分层知识融合网络(Hierarchical Knowledge Fusion Network,HKFN);3)对抗性知识正则化(Adversarial Knowledge Regularizer,AKR).在7个不同领域的语料库上进行的文本分类实验表明,本文所提模型与之前的模型相比,性能有显著提升.
Pre-trained language models like BERT excel at capturing general linguistic patterns but often un-derperform in domain-specific text classification due to a lack of structured knowledge.To enhance reasoning capabilities in specialized domains,knowledge injection has emerged as a mainstream approach.However,excessive knowledge fusion may distort the original semantics of sentences,leading to Knowledge Noise(KN).To address domain-specific knowledge gaps and mitigate the impact of KN in text classification,we propose a triple-enhanced BERT framework that integrates a Knowledge-Enhanced Dynamic Attention(KEDA),a Hierarchical Knowledge Fusion Network(HKFN),and an Adversarial Knowledge Regularizer(AKR).Experimental results on seven diverse domain-specific corpora demonstrate that our model signifi-cantly outperforms existing baselines.
孙豪;蒲亦非
四川大学计算机学院,成都 610065四川大学计算机学院,成都 610065
信息技术与安全科学
知识增强动态注意力分层知识融合知识正则化文本分类
knowledge-enhanceddynamic attentionhierarchical knowledge fusionknowledge regulariza-tiontext classification
《四川大学学报(自然科学版)》 2026 (3)
586-596,11
国家自然科学基金面上项目(62171303)中国兵器装备集团(成都)火控技术中心项目(非密)(HK20-03)国家重点研发项目(2018YFC0830300)
评论