基于输出激活引导的大模型通道级自适应稀疏度剪枝方法OA

Post-training pruning(PTP)has emerged as an efficient compression technique to address the challenges of limited computational resources and excessive memory footprint during the edge deployment of Large Language Models(LLMs).However,existing mainstream methods(e.g.,Wanda and SparseGPT)typically employ uniform layer-wise sparsity strategies,overlooking the significant heterogeneity in information contribution across different layers and channels.Moreover,their evaluation criteria predominantly focus on input-side intensity,making it difficult to identify high-energy static redundant channels,which leads to severe model performance degradation under high compression ratios.To address these limitations,this paper proposes OGAS,an Output-activation Guided Adaptive Sparsity pruning method at the channel level.First,a dual evaluation metric is constructed by integrating the output activation energy norm with the Peak-to-Average Power Ratio(PAPR)to accurately identify and protect sparse key features from the dimensions of response intensity and feature specificity.Second,a continuous mapping mechanism based on non-linear curvature is designed to achieve dynamic adaptive allocation of channel-level sparsity within a continuous space.Furthermore,a closed-loop optimization workflow is established by introducing the Golden Section Search algorithm to realize the automated layer-wise tuning of critical hyperparameters.Experimental results on mainstream open-source models,including LLaMA-3 and Mistral,demonstrate that at a 50%sparsity ratio,OGAS reduces the perplexity(PPL)of LLaMA-3.1-8B on the WikiText-2 dataset to 7.99,a significant improvement over the state-of-the-art first-order method Wanda(8.85).In common sense reasoning tasks,the average zero-shot accuracy reached 63.46%,representing a 1.6%improvement over Wanda.The results verify that OGAS effectively maintains the semantic understanding and logical reasoning capabilities of models after large-scale compression,exhibiting superior robustness and versatility across different model architectures.