面向成本收益协同优化的高质量数据准备方法OA
A cost-benefit collaborative optimization method for high-quality data preparation
针对实际机器学习(machine learning,ML)模型训练场景,数据准备环节资源利用率低、成本不可控及预算约束下优化不足等问题,构建了一种面向成本收益协同优化的数据准备整体框架CBAprep(cost-benefit-aware data preparation).采用数学规划方法,将数据预处理、清洗、转换等操作统一纳入成本建模与决策框架,并设计了面向数据管道的成本收益分析规划方法.实验结果表明,在有限预算条件下,CBAprep可实现更优的成本效益权衡,提升数据准备过程的整体性能与性价比.研究表明,将成本收益分析和优化的思想纳入数据准备优化策略,有助于实现资源的高效配置,并为复杂数据处理场景中的成本控制与性能优化提供参考.
In practical machine learning(ML)model training scenarios,data preparation is still constrained by low resource utilization efficiency,uncontrollable costs,and insufficient optimization under budget constraints.To address these problems,a holistic data preparation framework named CBAprep(cost-benefit-aware data preparation)was proposed,oriented toward cost-benefit collaborative optimization.Mathematical programming was adopted to incorporate data preprocessing,cleaning,transformation,and other operations into a unified cost modeling and decision-making framework,and a cost-benefit analysis and planning method for the overall pipeline was designed.Experimental results showed that CBAprep achieved a better trade-off between cost and performance under limited budget conditions,thereby improving the overall performance and cost-effectiveness of the data preparation process.The results indicate that incorporating cost-benefit analysis and optimization into data preparation strategies facilitates efficient resource allocation and provides useful support for cost control and performance optimization in complex data processing scenarios.
崔文萱;丁小欧;程海峰;钱泽凯;李勇;王宏志;王晨
哈尔滨工业大学计算学部,黑龙江 哈尔滨 150001哈尔滨工业大学计算学部,黑龙江 哈尔滨 150001哈尔滨工业大学计算学部,黑龙江 哈尔滨 150001哈尔滨工业大学计算学部,黑龙江 哈尔滨 150001哈尔滨工业大学学校办公室,黑龙江 哈尔滨 150001哈尔滨工业大学计算学部,黑龙江 哈尔滨 150001清华大学大数据系统软件国家工程实验室,北京 100084
信息技术与安全科学
数据治理数据准备资源调度成本收益
data governancedata preparationresource schedulingcost-benefit
《大数据》 2026 (5)
21-42,22
国家科技重大专项(No.2025ZD1607102)国家自然科学基金面上项目(No.62572151)黑龙江省自然科学基金优秀青年项目(No.YQ2024F005)黑龙江省青年科技人才托举工程项目(No.2025QNTJ001)中央高校基本科研业务费专项资金资助(No.HITAI4SE2026035) National Science and Technology Major Project of China(No.2025ZD1607102),The National Natural Science Foundation of China General Program(No.62572151),Heilongjiang Provincial Natural Science Foundation Excellent Young Scholars Program(No.YQ2024F005),Heilongjiang Young Scientific and Technological Talent Support Program(No.2025QNTJ001),The Fundamental Research Funds for the Central Universities(No.HITAI4SE2026035)
评论