基于Transformer的整数运算学习综述OA

Integer arithmetic serves as the foundation of symbolic computation and mathematical reasoning.While Transformer-based large language models have significantly advanced natural language processing,their competence in mathematical rea-soning increasingly stands as a key criterion for assessing model performance.Although several surveys have systemically sum-marized researches on mathematical reasoning with Transformer,a detailed analysis of progress in learning integer arithmetic using such model remains lacking.To provide a reference for further investigation,this paper firstly introduced technologies in Transformer and large language model,followed by a systematic review of studies on integer arithmetic learning based on Transformer,where this paper partitioned existing works into those employing models trained from scratch and those utilizing pretrained models by technological approaches.It then subdivided the latter into in-context learning and finetuning.Further-more,the paper also presented a comparison and relevance analysis,in which this paper investigated learning mechanisms of integer arithmetic and then reported a summary and analysis of relevant datasets.Finally,the paper discussed current challen-ges and potential research directions,with the aim of informing future research towards more advanced computation and reaso-ning tasks.