跨模态深度伪造检测技术综述OA

Review of Cross-modal Deepfake Detection Technologies

中文摘要英文摘要

生成对抗网络、扩散模型及多模态生成技术快速发展,使图像、视频、语音类深度伪造内容的逼真度持续提升,给数字媒体真实性鉴别带来严峻挑战.围绕深度伪造检测研究,基于伪造内容在"生成—渲染—编码—传播"链路中的异常形成过程,构建融合信号层伪影与语义层不一致特征的综述分析框架.首先,从图像、视频、语音3类单模态出发,系统梳理基于频域统计、设备指纹、重采样痕迹、光照几何、生理信号与时序结构等线索的检测方法;其次,围绕跨模态场景,归纳时序失配、生物与物理约束破坏、语义与情感冲突等关键证据,总结时序对齐建模、联合约束建模、异常分布学习及多级融合实现路径;进一步分析现有基准数据集与评价指标,指出当前评测体系在长时序、开放场景与系统级安全评估方面存在不足.综述表明,深度伪造检测正由依赖模态内统计异常的判别模式,逐步转向面向跨模态一致性约束与风险敏感评估的综合检测范式.

With the rapid development of generative adversarial networks,diffusion models,and multimodal generation technologies,the realism of deepfake content in images,videos,and audio has been continuously improving,posing significant challenges to digital media authenticity verification.Focusing on research progress in deepfake detection,this paper constructs a comprehensive analytical framework that combines signal layer artifacts with semantic layer inconsistencies features,based on the anomalous formation process of deepfake content in the"generation-rendering-encoding-dissemination"chain.First,from the perspectives of image,video,and audio as single modalities,the study systematically reviews detection methods utilizing clues such as frequency domain statistics,device fingerprints,resampling traces,lighting geometry,physiological signals,and temporal structures.Second,for cross-modal scenarios,key evidence including temporal mismatches,violations of biological and physical constraints,and semantic/emotional conflicts is summarized,along with implementation paths for temporal alignment modeling,joint constraint modeling,anomaly distribution learning,and multi-level fusion.Furthermore,an analysis of existing benchmark datasets and evaluation metrics reveals shortcomings in current assessment systems regarding long temporal sequences,open scenarios,and system-level security evaluation.The review indicates that deepfake detection is transitioning from a discriminative model relying on intra-modal statistical anomalies to a comprehensive detection paradigm emphasizing cross-modal consistency constraints and risk-sensitive evaluation.

车召旭;周顺勇;谢乐惠;刘盈盈;梁蔚;林伟

四川轻化工大学 自动化与信息工程学院,四川 宜宾 644000||智能感知与控制四川省重点实验室,四川 宜宾 644000四川轻化工大学 自动化与信息工程学院,四川 宜宾 644000||智能感知与控制四川省重点实验室,四川 宜宾 644000四川轻化工大学 自动化与信息工程学院,四川 宜宾 644000||智能感知与控制四川省重点实验室,四川 宜宾 644000四川轻化工大学 自动化与信息工程学院,四川 宜宾 644000||智能感知与控制四川省重点实验室,四川 宜宾 644000四川轻化工大学 自动化与信息工程学院,四川 宜宾 644000||智能感知与控制四川省重点实验室,四川 宜宾 644000四川轻化工大学 自动化与信息工程学院,四川 宜宾 644000||智能感知与控制四川省重点实验室,四川 宜宾 644000

信息技术与安全科学

深度伪造检测信号层伪影语义一致性约束跨模态一致性多模态生成

deepfake detectionsignal layer artifactssemantic consistency constraintscross-modal consistencymultimodal generation

《四川轻化工大学学报(自然科学版)》 2026 (3)

103-118,16

国家自然科学基金项目(61801319)四川省科技厅项目(2024ZYD0265)四川轻化工大学科研项目(2023RC24)

10.11863/j.suse.2026.03.11

评论