一种基于权重重构的忆阻神经网络剪枝方法OA
A pruning method for RRAM neural networks based on weight reconstruction
电阻式随机存取存储器(Resistive Random Access Memory,RRAM)因具备存内计算能力,被认为是高效的神经网络加速器.剪枝技术通过去除冗余权重可有效压缩模型,从而节省基于 RRAM 的神经网络加速器的硬件资源.现有的针对 RRAM 的结构化剪枝方法因其过粗的剪枝粒度易导致精度下降,且普遍忽视了权重之间的数值规律,导致这类潜在冗余未能被利用,难以在保证精度的同时进一步提升模型压缩率与硬件效率.为此,本文提出一种基于权重重构的忆阻神经网络剪枝方法,使用基于整数缩放的权重重构策略提取并共享权重中的数值共性,同时舍弃对精度影响较小的数值部分,仅映射权重关键信息至 RRAM 交叉阵列进行网络推理,实现权重的压缩表示.随后,使用渐进式重训练机制,将被舍弃的信息作为引导信号逐步衰减引入,从而在保持模型压缩率和硬件效率的同时有效恢复模型精度.实验结果表明,与现有方法相比,本文方法在模型压缩率、面积效率与能效方面实现了最多1.2 倍、1.2倍与 1.3 倍的提升,且几乎不损失模型精度.
Resistive Random Access Memory(RRAM),with its inherent processing-in-memory capability,has emerged as an efficient hardware platform for neural network acceleration.Pruning techniques effectively compress neural networks by removing redundant weights,thereby reducing the hardware cost of RRAM-based accelerators.However,existing RRAM-oriented structured pruning methods often suffer from exces-sively coarse granularity,which can lead to accuracy degradation.Moreover,they typically neglect the nu-merical patterns shared among weights,leaving potential redundancy underexploited and limiting further im-provements in compression and hardware efficiency.To address these challenges,we propose a pruning method for RRAM neural networks based on Weight Reconstruction.Specifically,an integer scaling weight reconstruction strategy is designed to extract and share common numerical structures among weights,while discarding components with minimal impact on model accuracy.The essential weight information is then mapped onto the RRAM crossbar for network inference,achieving a compact weight representation.Further-more,a progressive retraining mechanism is introduced,where the discarded components are leveraged as guidance signals that are gradually attenuated to refine the model,thereby recovering accuracy while maintain-ing high compression and hardware efficiency.Experiments show that,compared to the state-of-the-art works,the proposed method achieves up to 1.2×,1.2×,and 1.3×improvements in compression rate,area efficiency,and energy efficiency,respectively,with negligible accuracy loss.
刘静;刘鹏;姚廉;武继刚
广东工业大学计算机学院,广州 510006广东工业大学计算机学院,广州 510006广东工业大学计算机学院,广州 510006广东工业大学计算机学院,广州 510006
信息技术与安全科学
电阻式随机存取存储器;神经网络;剪枝;模型压缩;神经网络加速器
RRAMNeural networkPruningModel compressionNeural network accelerator
《四川大学学报(自然科学版)》 2026 (2)
275-286,12
国家自然科学基金(62374047,62174038)计算机体系结构国家重点实验室开放课题(CLQ 202407)
评论