Benchmarking Audio Deepfake Detection Robustness In Real-world Communication Scenarios

Abstract

Existing Audio Deepfake Detection (ADD) systems often struggle to generalise effectively due to the significantly degraded audio quality caused by audio codec compression and channel transmission effects in real-world communication scenarios. To address this challenge, we developed a rigorous benchmark to evaluate the performance of the ADD system under such scenarios. We introduced ADD-C, a new test dataset to evaluate the robustness of ADD systems under diverse communication conditions, including different combinations of audio codecs for compression and packet loss rates. Benchmarking three baseline ADD models on the ADD-C dataset demonstrated a significant decline in robustness under such conditions. A novel Data Augmentation (DA) strategy was proposed to improve the robustness of ADD systems. Experimental results demonstrated that the proposed approach significantly enhances the performance of ADD systems on the proposed ADD-C dataset. Our benchmark can assist future efforts towar

Stats

• citations4
• S2 citations—
• github stars0
• HF likes0
• heat score5.24
• arxiv keyshi2025benchmarking

Related papers

• Adversarial Attacks On Audio Deepfake Detection: A Benchmark And Comparative Study (2025)0.00
• Towards Robust Audio Deepfake Detection: A Evolving Benchmark For Continual Learning (2024)0.00
• The Codecfake Dataset And Countermeasures For The Universally Detection Of Deepfake Audio (2024)10.97
• AUDETER: A Large-scale Dataset For Deepfake Audio Detection In Open Worlds (2025)0.00
• Xmad-bench: Cross-domain Multilingual Audio Deepfake Benchmark (2025)1.69
• SLIM: Style-linguistics Mismatch Model For Generalized Audio Deepfake Detection (2024)4.52
• Betray Oneself: A Novel Audio Deepfake Detection Model Via Mono-to-stereo Conversion (2023)10.04
• The Vicomtech Audio Deepfake Detection System Based On Wav2vec2 For The 2022 ADD Challenge (2022)14.06