基于组合Q网络的确定性策略梯度OA

In terms of the issues that value function estimation bias and unstable policy learning process often occur in the training phase of off-policy Actor-Critic deep reinforcement learning algorithm,a deterministic policy gradient based on combined Q-network is proposed.On the one hand,referring to the problem of value function estimation bias,a combined Q-network mechanism is built to perform the adaptive weighted combination of outputs of multiple Q-networks,thus obtaining a more accurate temporal difference(TD)target.In the process of estimating the value function,the weights can be adaptively adjusted based on the deviation between the Q-value estimation and the discount return.On the other hand,the stability of policy learning process is enhanced by constraining the L2 norm between the old and new policies.The effectiveness and superiority of the proposed algorithm are verified by the simulation results of continuous control tasks on MuJoCo platform.