设备端大语言模型研究综述OA

The development of large language models(LLMs)technology is pushing the boundaries of natural language processing and artificial intelligence.However,cloud-based deployment of LLMs faces significant challenges,including latency issues,privacy risks,network dependency,and high resource consumption.To address these issues,on-device LLMs have gradually become a research hotspot,aiming to localized deployment on resource-constrained devices.This shift enh-ances real-time responsiveness,strengthens data privacy,and improves energy efficiency.The article surveys the current state of research on on-device large language models by focusing on two dimensions:model lightweighting and system-level support.Firstly,it surveys current mainstream model compression methods,including quantization,pruning,knowledge distillation and low-rank decomposition.Secondly,it discusses model architecture innovations tailored for on-device scenarios,including mixture-of-experts approaches and hierarchical designs.Additionally,it covers key system-level technologies,such as lightweight inference engines,specialized AI chips for edge devices,and memory optimization strategies.Finally,the paper highlights the major challenges and outlines potential future directions for advancing on-device large language models.