首页|期刊导航|自动化学报|基于分层策略强化学习的多类型流量差异化路由优化

基于分层策略强化学习的多类型流量差异化路由优化OA

Differentiated Routing Optimization for Multi-type Traffic Based on Hierarchical Policy Reinforcement Learning

中文摘要英文摘要

路由是优化网络资源分配的重要方法.然而,传统路由算法依赖静态策略优化单一服务质量指标,难以应对多类型流量爆发性增长下的差异化需求.尽管深度强化学习为动态网络环境下的路由优化提供了新思路,现有方法仍缺乏对流量类型的精细化感知能力,无法灵活调整路由策略.为此,本文针对不同类型流量的差异化路由需求,设计一种基于分层策略强化学习的流量感知路由算法.首先,引入流量分类模块,实现对不同流量差异化业务需求的精细感知.其次,利用图卷积网络对网络拓扑进行高效建模,并在此基础上设计分层决策网络以及差异化奖励函数,引导智能体生成自适应路由决策,实现对各流量类别路由策略的动态调整.同时,在演员-评论家框架中引入全局注意力机制,增强智能体对网络状态时空依赖关系的建模能力,并通过广义优势估计和近端策略优化算法提升训练的效率与稳定性.最后,在多种拓扑网络上验证了所提算法的有效性.

Routing is an important method for optimizing network resource allocation.However,traditional rout-ing algorithms rely on static strategies to optimize single quality of service metrics,making it difficult to address the differentiated requirements of explosive growth in multi-type traffic.Although deep reinforcement learning has provided new ideas for routing optimization in dynamic network environments,existing methods still lack fine-grained perception of traffic types and cannot flexibly adjust routing strategies.To this end,this paper designs a traffic-aware routing algorithm based on hierarchical policy reinforcement learning for the differentiated routing re-quirements of different traffic types.First,a traffic classification module is introduced to achieve fine-grained per-ception of the differentiated service requirements of different traffic.Second,graph convolutional networks are used to efficiently model the network topology,based on which a hierarchical decision network and a differentiated re-ward function are designed to guide the agent to generate adaptive routing decisions and realize dynamic adjust-ment of routing strategies for each traffic category.Meanwhile,a global attention mechanism is introduced into the actor-critic framework to enhance the agent's ability to model the spatio-temporal dependency of network states,and the training efficiency and stability are improved through generalized advantage estimation and proximal policy optimization algorithms.Finally,the effectiveness of the proposed algorithm is verified on various network topologies.

赵之栩;刘坤;王璐瑶;夏元清

北京理工大学自动化学院自主智能无人系统全国重点实验室 北京 100081北京理工大学自动化学院自主智能无人系统全国重点实验室 北京 100081北京理工大学自动化学院自主智能无人系统全国重点实验室 北京 100081北京理工大学自动化学院自主智能无人系统全国重点实验室 北京 100081

多类型流量深度强化学习注意力机制差异化路由QoS优化

multi-type trafficdeep reinforcement learningattention mechanismdifferentiated routingquality of service optimization

《自动化学报》 2026 (4)

709-723,15

10.16383/j.aas.c250413

评论