基于时间与图嵌入强化学习的含5G基站储能配电网安全调度策略OA

With the large-scale deployment of 5G base stations,a number of base station energy storage systems have been integrated into the power grid.These storage devices possess a certain degree of dispatch flexibility and have the potential to facilitate the consumption of renewable energy.How to make full use of these energy storage resources in distribution network scheduling has become an important issue.Existing scheduling algorithms,however,suffer from long calculation times,resulting in poor real-time performance.In addition,traditional algorithms rely on precise mathematical models and are easily affected by environmental uncertainties,which leads to inaccurate solutions that deviate from the optimal solution.To address these challenges,this paper proposes a real-time scheduling algorithm based on reinforcement learning(RL),tailored to the characteristics of 5G base station energy storage,leveraging the fast decision capability of RL to handle the scheduling challenges. Firstly,based on the economic operation requirements of the distribution network,multiple cost components are considered,including the cost of purchasing electricity from the main grid,the cost of abandoning renewable energy,and the operating cost of the 5G base station energy storages.Taking into account the safety capacity constraints of 5G base station energy storages and physical constraints of distribution network,a mathematical optimization model for real-time scheduling is established.This model is then transformed into a Markov decision process(MDP),providing the foundation for an RL-based solution framework. Secondly,considering the characteristics of state information during the scheduling process,different types of information are processed separately.For temporal information,a novel Time2Vec(T2V)architecture is introduced to extract time-related features.For the graph-structured information,a graph convolutional network(GCN)is employed for feature extraction.By integrating these modules,the traditional deep deterministic policy gradient(DDPG)algorithm is improved,resulting in the proposed T2V-GCN-DDPG algorithm,which enhances the ability of agent to extract hidden features from the state,thereby improving decision-making performance.In addition,to ensure the safety of actions from agent,a safety constraint layer based on quadratic programming(QP)is designed to fine-tune the charging and discharging power of the base station energy storage.The QP model aims to minimize the magnitude of action adjustment while keeping the actions within the safe range. Finally,to validate the effectiveness of the proposed algorithm,experiments were conducted in modified IEEE 33-bus and 141-bus distribution networks incorporating 5G base station energy storage systems.By comparing the results with those obtained from intraday rolling optimization,proximal policy optimization(PPO),and behavior cloning,it was demonstrated that the proposed algorithm achieves superior decision-making performance.It effectively reduces the deviation in actions caused by environmental uncertainties and generates actions closer to the optimal solution.Moreover,comparisons of the algorithm before and after the proposed improvements further confirm the effectiveness of the enhancement method.Simulation results also indicate that the proposed algorithm,while ensuring the safety of decision actions,can reduce power grid operating costs and improve overall economic performance.Therefore,the proposed algorithm provides a reliable and efficient solution for the safe and economical operation of the power grid.