基于多目标强化学习的太阳能无人机航迹规划OA

There is a significant coupling between the influencing factors of high-altitude long-endurance solar-powered UAVs in harvesting solar energy and gradient wind energy,and optimizing the harvesting efficiency of these two types of energy simultaneously often leads to conflicts.To address this issue,this study proposes a trajectory planning method based on multi-objective reinforcement learning.This method adopts the multi-objective Soft Actor-Critic(SAC)algorithm based on the multi-objective Markov decision process,combines the UAV's energy harvesting power and energy consumption power into a reward vector,and adds randomly generated weights in each update step.The converged trained policy network can output thrust,attack angle,and bank angle commands based on flight information and a given weight vector,enabling the generation of a set of energy-optimal trajectory solutions within the weight space.Simulation results show that compared with the minimum energy consumption strategy and the strategy based on the conventional single-objective SAC algorithm,this method consistently achieves better energy optimiza-tion efficiency and can adaptively respond to weight changes of energy objectives.Compared with the offline optimized trajectory solution set based on the Non-dominated Sorting Genetic Algorithm Ⅱ,this method achieves a hypervolume of the trajectory solution set reaching 90.07%of the former while maintaining excellent real-time performance.In addi-tion,this method also demonstrates a certain degree of generalization ability and can adapt to new untrained wind fields.