首页|期刊导航|水下无人系统学报|基于延迟鲁棒多智能体强化学习的协同围捕方法

基于延迟鲁棒多智能体强化学习的协同围捕方法OA

A Cooperative Encirclement Method Based on Delay-Robust Multi-Agent Reinforcement Learning

中文摘要英文摘要

针对多个无人水下航行器(UUV)协同作业时因水声通信机制导致的延迟观测问题,提出了一种基于延迟鲁棒多智能体强化学习的协同围捕方法.首先,在常用通信场景下分析延迟观测的产生原理,利用二维纳维-斯托克斯方程对湍流场进行建模,搭建仿真环境,模拟真实任务场景;然后,提出延迟鲁棒多智能体强化学习方法,并详细介绍各组成模块.在此基础上设计围捕任务的奖励函数、不同网络的模型结构以及训练方法,完成不同任务和延迟条件下的消融实验.实验结果表明,文中方法能有效应对延迟观测问题,在不同程度的延迟下均保持较好表现,部分任务中接近无时隙情况的理论性能上限.消融实验证明了所提方法中各组成模块在对抗延迟观测时的有效性,为延迟观测下的多 UUV 协作策略研究提供了新的理论支撑与技术路径.

To address delayed observations caused by acoustic communication in cooperative operations of multiple unmanned undersea vehicles(UUVs),a cooperative encirclement method based on delay-robust multi-agent reinforcement learning was proposed.First,the mechanism of delayed observations was analyzed under common communication scenarios.Second,a turbulent flow field was modeled using the two-dimensional Navier-Stokes equations to construct a simulation environment that reflects realistic task settings.Then,a delay-robust multi-agent reinforcement learning method was proposed,and its constituent modules were described in detail.On this basis,the reward functions for the encirclement task,the network architectures,and the training procedures were designed,and ablation studies were conducted under different tasks and delay conditions.Experimental results demonstrate that the proposed method effectively handles delayed observations and maintains strong performance under varying delay levels,approaching the theoretical upper bound of the no-delay case in some tasks.Furthermore,the ablation results verify the effectiveness of each module in mitigating delayed observations,providing new theoretical support and a practical methodology for multi-UUV cooperative strategies under delayed observations.

付松琛;白乐天;赵少靖;蒙傲萌;梁宏;黎塔

中国科学院声学研究所 语音与智能信息处理实验室,北京,100190||中国科学院大学,北京,100049中国科学院声学研究所 语音与智能信息处理实验室,北京,100190||中国科学院大学,北京,100049中国科学院声学研究所 语音与智能信息处理实验室,北京,100190||中国科学院大学,北京,100049中国科学院声学研究所 语音与智能信息处理实验室,北京,100190||中国科学院大学,北京,100049中国科学院声学研究所 语音与智能信息处理实验室,北京,100190||中国科学院大学,北京,100049中国科学院声学研究所 语音与智能信息处理实验室,北京,100190||中国科学院大学,北京,100049

军事科技

无人水下航行器协同围捕多智能体强化学习延迟观测

unmanned undersea vehiclecooperative encirclementmulti-agent reinforcement learningdelayed observation

《水下无人系统学报》 2026 (4)

726-740,15

10.11993/j.issn.2096-3920.2025-0173

评论