首页|期刊导航|计算机应用研究|交错模态思维链奖励机制的细粒度图像生成

交错模态思维链奖励机制的细粒度图像生成OA

Fine-grained image generation via interleaved multimodal chain-of-thought reward

中文摘要英文摘要

复杂细粒度语义对齐是当前文生图模型面临的核心挑战之一,尤其在属性绑定、数量关系与空间结构等组合语义场景中表现不佳.现有方法往往从特定模型结构出发,依赖高成本数据训练与推理,难以进行稳定有效的语义修正与跨范式的统一.针对上述问题进行了研究,提出基于交错模态思维链推理机制的统一语义对齐奖励微调框架ICOTR,利用多模态大模型的图文理解能力构建自适应多维语义奖励反馈,对扩散模型与自回归模型进行一致的语义引导优化.在高语义复杂度评测基准T2I-CompBench++上的实验中,ICOTR将SDXL的综合性能由0.411 4提升至0.496 5,将Show-o的综合性能由0.381 5提升至0.520 2.结果表明,ICOTR在整体性能及多个关键子任务上均显著优于现有方法,可有效提升不同生成范式下的细粒度图像生成能力.

Fine-grained semantic alignment remains a major challenge for current text-to-image generation models,especially in compositional scenarios involving attribute binding,numeracy,and spatial structure.Existing methods often rely on specific model architectures and high-cost training or inference,which limits stable semantic correction and unifies optimization across paradigms.To address this issue,this study proposed ICOTR,a unified semantic alignment reward fine-tuning framework based on an interleaved multimodal chain-of-thought reasoning mechanism.ICOTR used the image-text understanding capabili-ty of multimodal large language models to build adaptive multi-dimensional semantic reward feedback,and applied consistent semantic guidance to both diffusion and autoregressive models.Experiments on the high-semantic-complexity benchmark T2I-CompBench++show that ICOTR improved the overall performance of SDXL from 0.411 4 to 0.496 5 and that of Show-o from 0.381 5 to 0.520 2.The results indicate that ICOTR consistently outperforms existing methods in overall performance and se-veral key subtasks,and effectively improves fine-grained image generation across different paradigms.

郑凯;周远远;徐林莉

中国科学技术大学 计算机科学与技术学院,合肥 230027中国科学技术大学 信息科学实验中心,合肥 230027中国科学技术大学 计算机科学与技术学院,合肥 230027

信息技术与安全科学

细粒度文生图思维链奖励微调多模态大模型

fine-grained text-to-image generationchain-of-thoughtreward fine-tuningmultimodal large language model

《计算机应用研究》 2026 (9)

2589-2596,8

国家自然科学基金资助项目(62276245)

10.19734/j.issn.1001-3695.2026.02.0023

评论