首页|期刊导航|Computers, Materials & Continua|A Prosody-Guided Multi-Stream Framework for Universal Detection of AI-Synthesized Speech across Codec and Vocoder Domains

A Prosody-Guided Multi-Stream Framework for Universal Detection of AI-Synthesized Speech across Codec and Vocoder DomainsOA

中文摘要

Recent advancements in AI-synthesized speech have resulted in highly realistic deepfake audio,posing severe threats to authentication systems and digital media trust.Existing detection models struggle to generalize across diverse synthesis methods,especially those involving neural codec-based Audio Language Models(ALMs).In this work,we propose UniTector++,a novel prosody-aware,multi-stream detection architecture that generalizes across vocoder-and codec-based synthesis.UniTector++incorporates three complementary streams—Whisper-based semantic embeddings,high-level prosodic features,and codec artifact representations—fused through a Multi-Domain Adaptive Graph Attention Fusion(MAGAF)module.Furthermore,an Emotion-Consistency Verification Module(ECVM)reinforces alignment between speech style and prosodic content,and a Universal Adversarial Robustness(UAR)head improves resistance against adversarial attacks.Evaluated on three benchmark datasets—ASVspoof2021,PolyFake,and Codecfake—UniTector++achieves state-of-the-art performance with average Equal Error Rate(EER)of 0.57%under unseen synthesis scenarios,outperforming competitive baselines by a relative margin of 28%.Our results demonstrate the model’s superior generalization,interpretability,and robustness,offering a significant advancement in universal deepfake speech detection.

Akmalbek Abdusalomov;Mukhriddin Mukhiddinov;Fakhriddin Abdirazakov;Alpamis Kutlimuratov;Nodira Alimova;Ilyos Kalandarov;Ayhan Istanbullu;Rashid Nasimov;Young-Im Cho

Department of Computer Engineering,Gachon University,Seongnam-si,Republic of KoreaDepartment of Industrial Management and Digital Technologies,Nordic International University,Tashkent,Uzbekistan Department of Artificial Intelligence,Tashkent University of Information Technologies Named after Muhammad Al-Khwarizmi,Tashkent,UzbekistanDepartment of Computer Systems,Tashkent University of Information Technologies Named after Muhammad Al-Khwarizmi,Tashkent,UzbekistanDepartment of Applied Informatics,Kimyo International University in Tashkent,Tashkent,UzbekistanDepartment of Information Processing and Control Systems,Tashkent State Technical University,Tashkent,UzbekistanDepartment of Automation and Control,Navoi State University of Mining and Technologies,Navoi,UzbekistanDepartment of Computer Engineering,Faculty of Engineering,Balikesir University,Balikesir,TurkeyDepartment of Artificial Intelligence,Tashkent State University of Economics,Tashkent,UzbekistanDepartment of Computer Engineering,Gachon University,Seongnam-si,Republic of Korea

信息技术与安全科学

Deepfake speech detectionprosody analysisneural codec artifactswhisper modelmulti-stream fusionemotion-consistency verificationAI-synthesized speechspoofing detection

《Computers, Materials & Continua》 2026 (7)

P.2247-2267,21

supported by the Ministry of Trade,Industry and Energy and implemented by the Korea Institute for Advancement of Technology.The project includes Development of an International Standardization and Sustainability Integration Framework for AI Industry Internalization and Global Competitiveness Enhancement(RS-2025-07372968).

10.32604/cmc.2026.080444

评论