首页|期刊导航|电子科技|基于空间通道注意力的深度强化学习交通信号控制

基于空间通道注意力的深度强化学习交通信号控制OA

Traffic Signal Control by Deep Reinforcement Learning Based on Spatial Channel Attention

中文摘要英文摘要

现有基于强化学习的交通信号控制方法直接将预定义的交通状态输入策略网络,易导致状态信息冗余并且提取困难,从而影响智能体的策略输出.针对该问题,文中提出了一种结合空间和通道注意力机制的深度强化学习交通信号控制模型.在四维卷积神经网络中引入卷积块注意力模块,所提模型能够建模空间状态特征图和通道间的相互依赖关系,从而提升特征质量,优化信号控制效果.针对不同车流情况,设置不同状态特征维度并送入各自独立的经验池,在Q网络学习过程中同时考虑两个经验池的样本,以此保证在不同流量条件下输出最优策略.仿真结果表明,在SU-MO(Simulation of Urban Mobility)仿真平台上,相较于其他基准算法,所提模型在九路口交通网络的平峰和高峰条件下的车辆平均等待时间、平均速度和平均排队长度等指标的性能更优.

The existing traffic signal control methods based on reinforcement learning directly input the pre-defined traffic states into the policy network,which easily leads to redundant state information and difficulty in ex-traction,thereby affecting the policy output of the agent.For this problem,a deep reinforcement learning traffic sig-nal control model combining spatial and channel attention mechanisms is proposed.By introducing the convolution-al block attention module into the four-dimensional convolutional neural network,the proposed model can model the spatial state feature map and the interdependence between channels,thereby improving the feature quality and optimizing the signal control effect.For different traffic flow conditions,different state feature dimensions are set and sent to their respective independent experience pools.During the Q-network learning process,samples from both experience pools are considered simultaneously to ensure the output of the optimal strategy under different traf-fic conditions.The simulation results show that on the SUMO(Simulation of Urban Mobility)simulation platform,compared with other benchmark algorithms,the proposed model has better performance in terms of average waiting time,average speed and average queue length of vehicles under off-peak and peak conditions of the Nine-road in-tersection traffic network.

焦明怡;叶宝林;王翔;高慧敏;张一嘉

浙江理工大学 信息科学与工程学院,浙江 杭州 310018||嘉兴大学 信息科学与工程学院,浙江 嘉兴 314001浙江理工大学 信息科学与工程学院,浙江 杭州 310018||嘉兴大学 信息科学与工程学院,浙江 嘉兴 314001浙江嘉兴数字城市实验室有限公司,浙江 嘉兴 314001浙江理工大学 信息科学与工程学院,浙江 杭州 310018||嘉兴大学 信息科学与工程学院,浙江 嘉兴 314001浙江理工大学 信息科学与工程学院,浙江 杭州 310018

信息技术与安全科学

智能交通信号控制注意力机制深度强化学习深度对抗Q网络奖励函数经验回放SUMO自适应控制马尔科夫博弈

intelligent traffic signal controlattention mechanismdeep reinforcement learningdeep adver-sarial Q-networkreward functionexperience replaySUMOadaptive controlMarkov game

《电子科技》 2026 (9)

67-77,11

浙江省自然科学基金(LTGS23F030002)嘉兴市应用性基础研究项目(2023AY11034)Natural Science Foundation of Zhejiang(LTGS23F030002)Jiaxing Applied Basic Research Project(2023AY11034)

10.16180/j.cnki.issn1007-7820.2026.09.009

评论