气象超算系统资源数据管理与可视化技术研究OA

The China Meteorological Administration(CMA)has established large-scale supercomputing and intelligent computing systems distributed in Beijing,Inner Mongolia,and Shaanxi.These systems provide high-performance computing services for meteorological numerical forecasting operations.To enhance the management of diverse and heterogeneous computing power systems,improve system resource utilization,and complete the solutions of resource monitoring and metric for systems,end-users and applications,it is necessary to effectively manage the vast amounts of data related to these system resources.We explain the definitions and management methodologies of meteorological supercomputing resources.A unified metric model for resources of supercomputing and intelligent computing systems is proposed,and usage algorithm and charging algorithm are designed by the model.Based on the research,the solutions to monitoring,collection,storage,metric,charging and visualization of system resources data are proposed.The scheme leverages a distributed data management architecture inspired by the ELK(Elasticsearch,Logstash,Kibana)stack.System and job data are collected from Slurm,with data processing and storage facilitated by the Elasticsearch and Logstash.Data visualization is achieved through the Kibana.Finally,some actual cases are introduced in order to explain the effect in system management of data visualization.