首页|期刊导航|四川大学学报(自然科学版)|基于动态-分层-对抗协同优化的知识增强BERT文本分类模型

基于动态-分层-对抗协同优化的知识增强BERT文本分类模型OA

Classification:A triple enhancement framework combining dynamic,hierarchical,and adversarial mechanisms

中文摘要英文摘要

像BERT这样的预训练语言模型擅长捕捉通用语言模式,但由于缺乏结构化知识,在特定领域的文本分类中表现不佳.为了使模型在特定领域拥有出色的推理能力,知识注入逐渐成为主流.然而,过度的知识融合会改变句子的正确含义,导致知识噪声(Knowledge Noise,KN)问题.为了克服领域特定知识缺口和知识噪声在文本分类领域的影响,本文提出了一个三重增强的BERT框架,具体包括:1)知识增强动态注意力(Knowledge-Enhanced Dy-namic Attention,KEDA);2)分层知识融合网络(Hierarchical Knowledge Fusion Network,HKFN);3)对抗性知识正则化(Adversarial Knowledge Regularizer,AKR).在7个不同领域的语料库上进行的文本分类实验表明,本文所提模型与之前的模型相比,性能有显著提升.

Pre-trained language models like BERT excel at capturing general linguistic patterns but often un-derperform in domain-specific text classification due to a lack of structured knowledge.To enhance reasoning capabilities in specialized domains,knowledge injection has emerged as a mainstream approach.However,excessive knowledge fusion may distort the original semantics of sentences,leading to Knowledge Noise(KN).To address domain-specific knowledge gaps and mitigate the impact of KN in text classification,we propose a triple-enhanced BERT framework that integrates a Knowledge-Enhanced Dynamic Attention(KEDA),a Hierarchical Knowledge Fusion Network(HKFN),and an Adversarial Knowledge Regularizer(AKR).Experimental results on seven diverse domain-specific corpora demonstrate that our model signifi-cantly outperforms existing baselines.

孙豪;蒲亦非

四川大学计算机学院,成都 610065四川大学计算机学院,成都 610065

信息技术与安全科学

知识增强动态注意力分层知识融合知识正则化文本分类

knowledge-enhanceddynamic attentionhierarchical knowledge fusionknowledge regulariza-tiontext classification

《四川大学学报(自然科学版)》 2026 (3)

586-596,11

国家自然科学基金面上项目(62171303)中国兵器装备集团(成都)火控技术中心项目(非密)(HK20-03)国家重点研发项目(2018YFC0830300)

10.19907/j.0490-6756.250213

评论