首页|期刊导航|水下无人系统学报|基于AMDQN的AUV遍历路径规划方法

基于AMDQN的AUV遍历路径规划方法OA

AUV Traversal Planning Method Based on AMDQN

中文摘要英文摘要

为了提高自主水下航行器(AUV)的遍历路径规划能力,文中提出了一种基于改进的蒙豪森深度 Q 网络(AMDQN)的 AUV 遍历路径规划算法.首先建立贴合实际场景的遍历规划环境,利用射线覆盖方法代替传统矩形栅格式建模,以此提升环境建模精度,同时约束 AUV 的可选动作集,限制其机动能力.在上述环境下对算法各组件进行设计:状态空间方面,将模糊的全局环境信息、以 AUV 为中心的高精度局部环境信息,以及 AUV 自身位置信息进行融合,对环境进行系统化描述;奖励函数方面,引入了规则惩罚项与边缘指导机制,使 AUV 能够沿环境边缘稳定航行,提高覆盖率;参数更新方面,在 DQN 算法的基础上设计自适应温度参数更新策略,同时结合多步奖励与竞争网络架构缓解训练方差;在训练过程中,对全连接层参数执行软重置操作,以缓解局部最优现象.仿真实验结果证明,相较于传统方法和未改进的强化学习算法,文中所提算法环境适应性更强,同时能实现更高的遍历覆盖率.

To improve the traversal path planning capability of autonomous undersea vehicle(AUV),this paper proposed an AUV traversal path planning algorithm based on an advanced Munchausen deep Q-network(AMDQN).First,the paper established a traversal planning environment suitable for practical scenarios.On the one hand,the ray coverage method was used instead of the traditional rectangular grid modeling method to improve modeling accuracy;on the other hand,the optional action set of the AUV was constrained to limit its maneuverability.Second,each component of the algorithm was carefully designed under the above environment.Specifically,in the state space design,the paper fused vague global environmental information,accurate local environmental information centered on the AUV,and the AUV's own position information to systematically represent the environment.In the reward function design,rule penalty and edge guidance were introduced,enabling the AUV to stably improve coverage along environmental edges.For the parameter update method,an adaptive temperature parameter update strategy was designed based on DQN,while the multi-step reward and a dueling network architecture were introduced to alleviate training variance.During the training process,a soft reset of fully connected layer parameters was adopted to mitigate the local optimum problem.The simulation results show that the proposed algorithm has stronger environmental adaptability and higher traversal coverage than traditional methods and unimproved reinforcement learning algorithms.

庞舟岐;林晓波;耿士程;郝程鹏

中国科学院声学研究所,北京,100190||中国科学院大学,北京,100049中国科学院声学研究所,北京,100190中国科学院声学研究所,北京,100190||中国科学院大学,北京,100049中国科学院声学研究所,北京,100190||中国科学院大学,北京,100049

军事科技

自主水下航行器路径规划强化学习改进的蒙豪森深度Q网络

autonomous undersea vehiclepath planningreinforcement learningadvanced Munchausen deep Q-network

《水下无人系统学报》 2026 (4)

687-700,14

10.11993/j.issn.2096-3920.2026-0028

评论