面向经验质量与效率协同优化的改进MADDPG围捕算法OA

To address training oscillations and slow policy convergence caused by poor experience quality and low utiliza-tion efficiency in multi-agent deep reinforcement learning(MARL)for cooperative pursuit tasks,an improved algorithm,STEP-MADDPG(sequence-temporal experience prioritized multi-agent deep deterministic policy gradient)is proposed to synergistically optimize experience quality and efficiency.It constructs a dual-layer optimization framework featuring a"sample denoising and sequence guidance"closed-loop mechanism for the synergistic optimization of experience quality and efficiency.In the experience quality enhancement layer,long short-term memory(LSTM)networks extract temporal dependencies from observation sequences in partially observable environments,while the input structure of value network(Critic)is reconstructed to align the hidden states of policy network(Actor).Furthermore,a ternary decoupled sampling(TDS)strategy mitigates sample distribution conflicts between the Actor and Critic.By integrating the loss-adjusted priori-tized mechanism(LAP)with inverse prioritized sampling,TDS filters high-confidence experiences for the Actor,reducing gradient variance and enhancing training stability.In the experience efficiency optimization layer,prioritized sequence experience replay(PSER)focuses the model on critical experience sequences.Concurrently,an adaptive mixed sampling mechanism dynamically adjusts sample weights for exploration and exploitation during policy updates.The distribution correction effect of this mechanism inversely stabilizes the input distribution of quality layer,reducing temporal feature extraction uncertainty.Simulations conducted in 2D pursuit scenarios involving static and dynamic targets and obstacles demonstrate that the proposed STEP-MADDPG algorithm significantly outperforms baseline algorithms in terms of con-vergence speed and pursuit success rate across all evaluated scenarios.In fully dynamic scenarios,convergence speed im-proves by 29%to 42%,and the pursuit success rate increases by 6%to 25%.