推理时间约束的结构化剪枝OA

Structured pruning compresses and accelerates neural networks by removing groups of weights in a structured manner.Most existing pruning methods target a predefined sparsity level,i.e.,pruning a fixed proportion of weights,rather than directly optimizing inference latency.However,the relationship between sparsity and inference time is highly nonlinear,making sparsity-targeted pruning methods unsuitable for deployment scenarios with explicit latency constraints.To address this issue,we propose a novel structured pruning method,termed inference time constrained pruning(ITCP),which automatically searches for a pruning scheme that satisfies a desired inference-time budget while minimizing accuracy degradation.Specifically,ITCP formulates latency-constrained pruning as a constrained optimization problem,where the objective is to maximize a performance score under a given inference-time constraint,and solves it efficiently using dynamic programming.In addition,a performance model is devel-oped to rapidly estimate the inference time of models at different sparsity levels.Experimental results on CIFAR-10,CIFAR-100,and ImageNet demonstrate that,under the same acceleration requirements,ITCP consistently achieves higher accuracy than baseline pruning strategies.