基于行为预测和策略融合的轨道博弈决策方法OA
A Decision Method for Orbital Game Based on Behavior Prediction and Strategy Fusion
轨道追逃博弈中逃逸策略的高度未知性与行为多样性,给追踪策略的泛化能力带来严峻挑战.深度强化学习虽可提升追踪星的博弈效能,但当逃逸策略偏离训练分布时,策略网络易产生次优甚至失效的决策.为此,提出一种基于行为预测和策略融合的轨道博弈决策方法.在训练阶段,首先采用"预测制导+人工势场法"构建多样化逃逸策略集.随后在传统演员−评论家训练框架的基础上,通过引入预测网络构建预测器−演员−评论家算法,针对每类逃逸策略分别训练以获得对应的追踪子策略.其中预测网络用于估计逃逸星动作,并通过预测结果与真实动作的相似性衡量子策略与未知逃逸策略的匹配度.在执行阶段,策略融合器以逃逸星历史动作与各追踪子策略的预测结果为输入,动态计算匹配度并选择最优子策略进行博弈决策.实验结果表明,预测网络能有效评估追踪子策略对未知逃逸策略的适应性,策略融合器可显著提升追踪星面对多样化逃逸策略的泛化能力与可靠性.
The high uncertainty and behavioral diversity of evasion strategies in the orbital pursuit-evasion game pose significant challenges to the generalization capability of pursuit strategies.Although deep reinforcement learn-ing can enhance the pursuer's performance,the policy network often produces suboptimal or even invalid decisions when facing evasion strategies that deviate from the training distribution.To address this issue,this paper pro-poses a decision method for orbital game based on behavior prediction and strategy fusion,named predictor-actor-critic with fusion.During the training phase,a set of diverse evasion strategies is modeled using a prediction-guided approach combined with the artificial potential field method.Based on the traditional actor-critic framework,a pre-dictor-actor-critic algorithm is developed by introducing a prediction network,and a corresponding pursuit sub-policy is trained for each type of evasion strategy.The prediction network estimates the evader's actions,and the similarity between predicted and actual actions is used to quantify the matching degree between each sub-policy and the unknown evasion strategy.During the execution phase,the fusion module takes the evader's historical ac-tions and pursuit sub-policies'prediction outputs as input,dynamically evaluates matching degree,and selects the most appropriate sub-policy for decision-making.Experimental results demonstrate that the prediction network ef-fectively evaluates the adaptability of sub-policy to unknown evasion strategies,and the fusion module significantly enhances the generalization capability and reliability of the pursuer when confronted with diverse evasion strategies.
王英杰;袁利;黄煌;耿远卓
北京控制工程研究所 北京 100094||空间智能控制技术全国重点实验室 北京 100094空间智能控制技术全国重点实验室 北京 100094||中国空间技术研究院 北京 100094北京控制工程研究所 北京 100094||空间智能控制技术全国重点实验室 北京 100094北京控制工程研究所 北京 100094||空间智能控制技术全国重点实验室 北京 100094
轨道追逃博弈深度强化学习行为预测策略融合
orbital pursuit-evasion gamedeep reinforcement learningbehavior predictionstrategy fusion
《自动化学报》 2026 (3)
451-462,12
国家自然科学基金(62303047,U21B6001),空间智能控制技术全国重点实验室开放基金课题(2024-CXPT-GF-JJ-012-05)资助Supported by National Natural Science Foundation of China(62303047,U21B6001)and National Key Laboratory of Space In-telligent Control(2024-CXPT-GF-JJ-012-05)
评论