大语言模型作文评价反馈质量的实证分析OA

At present,large language model(LLM)povide an optimized opportunity for intelligent essay evaluation with their with advanced text processing capabilities.However,their feedback quality and semantic analysis ability remain insufficient.Issues such as superficial theme and over-polishing often reveal that LLM feedback cannot yet meet the core evaluation requirements of Chinese essay focusing on theme and expression.Therefore,it is necessary to evaluate their feedback quality.Therefore,this paper first developed a quality evaluation standard for LLM essay evaluation feedback through literature review and expert consultation.Ten LLM were used to evaluate 50 narrative and 50 argumentative essays respectively according to the given essay scoring criteria.Then,the paper scored the LLM feedback based on the standard,calculated inter-rater reliability,and conducted descriptive analysis and qualitative summary of the results.The study found out that LLM's feedback had room for improvement in stylistic stability,richness and in-depth understanding;LLM's feedback lacked teacher-like abilities in guiding thinking,critical evaluation and judging linguistic literary grace;LLM was good at providing feedback on theme,material,structure,emotion and ideology;LLM's feedback showed a certain degree of disciplinary professionalism.Finally,suggestions were put forward based on the findings,aiming to provide a reference for promoting the in-depth integration of LLM and Chinese language essay evaluation.