FlexCheck:融合异构存储的高效多级检查点框架OA

Multi-level checkpointing technology can effectively mitigate the issue of excessive applica-tion process blocking time caused by checkpoints and has been widely applied in the field of high performance computing(HPC).This technology initially writes check points to the local memory of compute nodes and subsequently flushes the data from local memory to more reliable storage,such as parallel file systems,in the background to enhance checkpoint reliability.However,as HPC systems continue to scale up,the size of applications increases,leading to a corresponding rise in checkpoint data volume.The capacity of local memory on compute nodes has thus become a bottleneck limiting the com-plete storage of checkpoint data.Additionally,in recent years,HPC systems have gradually introduced new types of storage,such as shared burst buffers,which the inherent framework of current multi-level checkpointing technologies cannot effectively leverage.To address these issues,a multi-level check-pointing framework named FlexCheck is proposed.This framework integrates heterogeneous storage re-sources in HPC systems,allowing checkpoint data to be distributed and stored across multiple types of heterogeneous storage rather than being confined solely to the local memory of compute nodes,effec-tively resolving the capacity bottleneck.Moreover,FlexCheck optimizes the application of shared burst buffers in checkpoint scenarios.Experimental evaluations demonstrate that when the checkpoint size reaches 2 048 GiB,FlexCheck only increases the execution time of application processes by 4.97%.