首页|期刊导航|Computers, Materials & Continua|A Feature-Adaptive Knowledge Distillation Framework for Efficient Offline-to-Online Reinforcement Learning

A Feature-Adaptive Knowledge Distillation Framework for Efficient Offline-to-Online Reinforcement LearningOA

中文摘要

Deep reinforcement learning(DRL)has gained significant attention as an essential technology for constructing intelligent agents capable of handling high-dimensional visual observations in complex control environments.With the rapid development of knowledge transfer paradigms,reincarnating reinforcement learning(RRL)has emerged as a promising approach to accelerate policy convergence and alleviate the inefficiency of traditional tabula rasa training by reusing pre-trained teacher policies.However,existing RRL approaches primarily focus on improving knowledge transfer efficiency,while how student networks adaptively regulate and selectively utilize inherited representations during the teacher–student transition remains underexplored.As a result,student agents may indiscriminately inherit environmental background noise and suboptimal representations from the teacher,leading to suboptimal policy convergence and severe performance plateaus.In this study,we introduce a novel feature-adaptive knowledge distillation framework,named FA-QDagger,to address the critical dilemma of selective inheritance.Distinct from existing attention-augmented value networks,our approach embeds the Squeeze-and-Excitation(SE)channel recalibration mechanism directly into the teacher-student distillation loop as a dynamic information bottleneck.Furthermore,through targeted recalibration operations,it adaptively amplifies task-critical decision signals while robustly filtering out the teacher’s suboptimal prior knowledge based on varying environmental states.The proposed approach is evaluated on high-dimensional visual control benchmarks across Atari 2600 environments under restricted offline and online interaction budgets.Experimental results on five representative Atari 2600 environments demonstrate that FA-QDagger consistently improves asymptotic performance over representative RRL baselines during online fine-tuning while maintaining the jump-start advantage of teacher-guided distillation.In addition,visual interpretability analysis using saliency maps confirms that the attention-enhanced agent develops more concentrated visual focus on decision-critical regions,indicating improved feature decoupling and robustness against environmental interference.

Baoping Tian;Zhuxiao Wang;Jiahao Xue;Hong Wang;Ying Zhang;Yun Ju

Department of Computer Science and Technology,School of Control and Computer Engineering,North China Electric Power University,Beijing,ChinaDepartment of Computer Science and Technology,School of Control and Computer Engineering,North China Electric Power University,Beijing,ChinaDepartment of Computer Science and Technology,School of Control and Computer Engineering,North China Electric Power University,Beijing,ChinaDepartment of Computer Science and Technology,School of Control and Computer Engineering,North China Electric Power University,Beijing,ChinaDepartment of Computer Science and Technology,School of Control and Computer Engineering,North China Electric Power University,Beijing,ChinaDepartment of Computer Science and Technology,School of Control and Computer Engineering,North China Electric Power University,Beijing,China

信息技术与安全科学

Deep reinforcement learningreincarnating reinforcement learningknowledge distillationoffline-to-online learningchannel attentionintelligent agent

《Computers, Materials & Continua》 2026 (11)

P.1721-1736,16

supported by theNationalNatural Science Foundation ofChina(No.52078212,No.62476086)the Fundamental Research Funds for the Central Universities of Ministry of Education of China of North China Electric Power University(Project No.2025JG004)State Grid Corporation of China Company Science and Technology Project Research(No.SGSXDKooHLJS2500142)Beijing Natural Science Foundation(L251012).

10.32604/cmc.2026.085661

评论