多模态语义通信技术综述OA

A Review of Multimodal Semantic Communication

中文摘要英文摘要

随着6G网络与人工智能的深度融合,传统比特精确复现的通信范式已难以满足智能任务对高效、低时延与高鲁棒交互的需求,多模态语义通信作为"传递意义"的新型范式受到广泛关注.该技术从任务驱动协同、统一语义表示到生成式跨模态通信的演进路径与核心机制被予以系统综述:首先,立足于语义熵与信息瓶颈等底层理论基石,追溯通信体系向多模态语义交互的范式演进脉络;其次,在此基础上,进一步梳理文本、图像、音视频等不同模态的语义建模与特征提取技术,揭示其共性与差异;然后,重点围绕任务驱动协同、语义驱动通信、生成式跨模态通信3类核心范式,深入剖析不同范式的设计权衡;最后,总结当前在语义度量、性能评估等方面面临的关键挑战,并展望面向6G智能网络的未来发展方向.

With the deep integration of 6G networks and artificial intelligence,the traditional communication paradigm of precise bit reproduction is no longer sufficient to meet the demands of intelligent tasks for efficient,low-latency,and robust interaction.Multimodal semantic communication,as a new paradigm for"meaning transmission",is widely recognized and investigated.In this paper,the evolution path and core mechanisms of this technology—from task-driven collaboration and unified semantic representation to generative cross-modal communication—are systematically reviewed.First,based on underlying theoretical foundations such as semantic entropy and information bottlenecks,the paradigm evolution of communication systems towards multimodal semantic interaction is traced.Second,based on this foundation,the semantic modeling and feature extraction technologies for different modalities such as text,images,and audio/video are further analyzed,and their commonalities and differences are revealed.Then,focusing on three core paradigms—task-driven collaboration,semantic-driven communication,and generative cross-modal communication—the design trade-offs of different paradigms are deeply examined.Finally,the key challenges currently faced in semantic measurement and performance evaluation are summarized,and the future development direction for 6G intelligent networks is envisioned.

段雪阳;陈明举;林志豪;罗扬铭;解晨

四川轻化工大学 自动化与信息工程学院,四川 宜宾 644000||智能感知与控制四川省重点实验室,四川 宜宾 644000四川轻化工大学 自动化与信息工程学院,四川 宜宾 644000||智能感知与控制四川省重点实验室,四川 宜宾 644000四川轻化工大学 自动化与信息工程学院,四川 宜宾 644000||智能感知与控制四川省重点实验室,四川 宜宾 644000四川轻化工大学 自动化与信息工程学院,四川 宜宾 644000||智能感知与控制四川省重点实验室,四川 宜宾 644000四川轻化工大学 自动化与信息工程学院,四川 宜宾 644000||智能感知与控制四川省重点实验室,四川 宜宾 644000

信息技术与安全科学

多模态语义通信跨模态融合生成式通信特征提取联合信源信道编码

multimodal semantic communicationcross-modal fusiongenerative communicationfeature extractionjoint source-channel coding

《四川轻化工大学学报(自然科学版)》 2026 (4)

86-102,17

四川省自然科学基金项目(2025ZNSFSC04772024NSFSC2042)四川轻化工大学研究生创新基金资助项目(Y2025072)

10.11863/j.suse.2026.04.08

评论