Containerlet: Software-Granular Container Layer for High-Performance Deployment of Machine Learning ApplicationsOA
Large-scale machine learning (ML) clusters deploy hundreds of models packaged as large container images, making scheduling and image pulling dominant contributors to deployment latency. Current systems adopt a hybrid-granularity strategy: they schedule using image-level metadata, reuse on-node layer caches, and lazily fetch content at the file or chunk level. This fragmentation leads to suboptimal cross-image reuse. We show that an intermediate unit, software granularity, can be uniformly applied at scheduling, caching, and pulling to improve deployment performance. We present Containerlet, a software-scoped layer that reorganizes standard images into one private layer plus multiple reusable per-software units that reside in compatible registries and are pulled by unmodified runtimes. Containerlet promotes cross-image reuse while bounding the number of units that the scheduler must consider. We design a file backtracking method to convert standard images into Containerlet images, which includes one private layer and multiple Containerlets. We also extend Kubernetes and container runtime to support software-granular scheduling, caching, and lazy pulling. End-to-end deployment experiments on diverse ML images and production cluster traces show consistent gains, reducing deployment overhead by 18% compared with state-of-the-art systems.
Feng-Zhi Li;Qing-Ru Xu;Tuo Zhou;Chen-Xi Wang;Yi-Fan Wang;Xiao-Hui Peng
Institute of Computing Technology,Chinese Academy of Sciences,Beijing 100190,China University of Chinese Academy of Sciences,Beijing 100149,ChinaInstitute of Computing Technology,Chinese Academy of Sciences,Beijing 100190,China University of Chinese Academy of Sciences,Beijing 100149,ChinaInstitute of Computing Technology,Chinese Academy of Sciences,Beijing 100190,China University of Chinese Academy of Sciences,Beijing 100149,ChinaInstitute of Computing Technology,Chinese Academy of Sciences,Beijing 100190,China University of Chinese Academy of Sciences,Beijing 100149,ChinaInstitute of Computing Technology,Chinese Academy of Sciences,Beijing 100190,China University of Chinese Academy of Sciences,Beijing 100149,ChinaInstitute of Computing Technology,Chinese Academy of Sciences,Beijing 100190,China University of Chinese Academy of Sciences,Beijing 100149,China
信息技术与安全科学
container deploymentcontainer imagecontainer schedulingmachine learning
《Journal of Computer Science & Technology》 2026 (3)
P.1024-1035,12
supported by the National Natural Science Foundation of China under Grant Nos. U23B2004 and 62072434.
评论