Xmad-bench: Cross-domain Multilingual Audio Deepfake Benchmark
2025 · Ioan-Paul Ciobanu, Andrei-Iulian Hiji, Nicolae-Catalin Ristea, et al.
Abstract
Recent advances in audio generation led to an increasing number of deepfakes, making the general public more vulnerable to financial scams, identity theft, and misinformation. Audio deepfake detectors promise to alleviate this issue, with many recent studies reporting accuracy rates close to 99%. However, these methods are typically tested in an in-domain setup, where the deepfake samples from the training and test sets are produced by the same generative models. To this end, we introduce XMAD-Bench, a large-scale cross-domain multilingual audio deepfake benchmark comprising 668.8 hours of real and deepfake speech. In our novel dataset, the speakers, the generative methods, and the real audio sources are distinct across training and test splits. This leads to a challenging cross-domain evaluation setup, where audio deepfake detectors can be tested "in the wild". Our in-domain and cross-domain experiments indicate a clear disparity between the in-domain performance of deepfake detectors
Authors
(none)
Tags
Stats
Related papers
- AUDETER: A Large-scale Dataset For Deepfake Audio Detection In Open Worlds (2025)0.00
- MLAAD: The Multi-language Audio Anti-spoofing Dataset (2024)13.34
- The Codecfake Dataset And Countermeasures For The Universally Detection Of Deepfake Audio (2024)10.97
- Towards Robust Audio Deepfake Detection: A Evolving Benchmark For Continual Learning (2024)0.00
- Benchmarking Audio Deepfake Detection Robustness In Real-world Communication Scenarios (2025)5.24
- Adversarial Attacks On Audio Deepfake Detection: A Benchmark And Comparative Study (2025)0.00
- Zero-day Audio Deepfake Detection Via Retrieval Augmentation And Profile Matching (2025)0.00
- Detection Of Cross-dataset Fake Audio Based On Prosodic And Pronunciation Features (2023)0.00