人机交互多轮对话情境下大语言模型的"迷失现象"OACHSSCD
"Lost in Conversation"Phenomenon of Large Language Models in Multi-Turn Dialogue Scenarios
[目的/意义]多轮对话已成为大语言模型执行复杂认知任务的重要形态,但主流评测以单轮任务为主,难以刻画多轮交互的输出分布风险.[方法/过程]基于"信息等价"原则,通过"指令分片—对齐检验—人工终审"流程构建受控对照数据集,选取4款大语言模型在五类任务上开展对照评估,以均值、分位边界与波动幅度刻画输出分布,并结合效应量、置信区间与双因素方差分析检验稳健性.[结果/结论]结果表明,多轮交互普遍降低平均性能并放大不确定性,呈现"上限稳定、下限断崖下移、波动扩大"的结构性特征;任务所需的上下文跨度与跨轮信息整合需求对衰退幅度具有显著调节作用,长文本处理与摘要生成等需要跨轮次维持全局语境的任务更易出现下限"击穿";受控实验进一步表明,交互频次是驱动性能不稳的关键变量,降低交互频次可显著缓解"迷失现象".研究从认知负荷理论与Transformer架构两个层面探讨产生机制,为多轮评测范式与人机协作设计提供量化证据.
[Purpose/Significance]Multi-turn dialogue has become a dominant interaction paradigm for large language models(LLMs)in complex cognitive tasks,yet mainstream evaluations rely on information-complete single-turn tasks that fail to capture the distributional risks introduced by multi-turn interactions.Although the"Lost in Conversation"pheno-menon has been identified in English settings,whether it generalizes to the Chinese context,which task types are most vul-nerable,and what variables drive the instability remain systematically unexplored.This study aimed to provide controlled,verifiable quantitative evidence on the existence,structural characteristics,and key drivers of this phenomenon in Chinese multi-turn dialogue.[Method/Process]This study constructed a controlled comparative framework grounded in an"infor-mation equivalence"principle,under which the single-turn and multi-turn conditions shared identical semantic content and constraints,with only the presentation mode varied.A total of 250 single-turn samples spanning five task types—read-ing comprehension,logical reasoning,text classification,long-text processing,and summarization—were curated from established Chinese benchmarks(CMRC 2018,CLUE,DuReader,Math23K,LCSTS),and each sample was converted into a paired multi-turn counterpart through an"instruction slicing—alignment verification—human final review"pipe-line.Four LLMs with distinct architectures and parameter scales(Gemini-2.5-Pro,Claude-3.7-Sonnet,Qwen1.5-72B-Chat,Yi-1.5-34B-Chat)each ran 10 repeated trials per instance under low-entropy decoding,yielding 20,000 scored observations evaluated by an LLM-as-a-Judge protocol supplemented with human spot-checks.The study assessed output distributions through mean performance,percentile bounds,and fluctuation ranges,and verified robustness via Cohen's d,95%confidence intervals,and two-way ANOVA.Controlled-effect experiments on 52 underperforming instances further isolated the independent contributions of temporal order,interaction frequency,and language context.[Result/Conclusion]The results confirm that multi-turn interaction in the Chinese context consistently reduces mean performance and amplifies uncertainty,exhibiting a structural pattern of"stable upper bounds,precipitous lower-bound drops,and expanded fluctua-tion."Mean scores decrease by 8.05 to 18.03 points across models,with all Cohen's d values exceeding 0.93.Task structure significantly moderates degradation severity:long-text processing and summarization suffer the most severe lower-bound breakdowns(P10 dropping to 43 and 52),while text classification remains comparatively resilient.Two-way ANOVA confirms that both main effects and their interaction reach high significance(P<0.0001).Controlled experiments reveal that interaction frequency is the dominant driver—compressing dialogue turns substantially recovers performance and narrows fluctuation across all task types.These findings point to a cascading mechanism of"cognitive load accumula-tion—attention dilution—state drift,"and suggest that designing fewer turns with denser instructions and embedding inter-mediate checkpoint mechanisms are essential practical strategies for reliable human-AI collaboration in high-stakes environments.
张卫东;李奉芮;胡文杰
吉林大学商学与管理学院,吉林 长春 130012吉林大学商学与管理学院,吉林 长春 130012吉林大学商学与管理学院,吉林 长春 130012
社会科学
大语言模型人机交互多轮对话迷失现象受控实验提示工程
large language modelshuman-computer interactionmulti-turn dialoguelost in conversation pheno-menoncontrolled experimentprompt engineering
《现代情报》 2026 (5)
28-40,13
评论