Hm-conformer: A Conformer-based Audio Deepfake Detection System With Hierarchical Pooling And Multi-level Classification Token Aggregation Methods

Abstract

Audio deepfake detection (ADD) is the task of detecting spoofing attacks generated by text-to-speech or voice conversion systems. Spoofing evidence, which helps to distinguish between spoofed and bona-fide utterances, might exist either locally or globally in the input features. To capture these, the Conformer, which consists of Transformers and CNN, possesses a suitable structure. However, since the Conformer was designed for sequence-to-sequence tasks, its direct application to ADD tasks may be sub-optimal. To tackle this limitation, we propose HM-Conformer by adopting two components: (1) Hierarchical pooling method progressively reducing the sequence length to eliminate duplicated information (2) Multi-level classification token aggregation method utilizing classification tokens to gather information from different blocks. Owing to these components, HM-Conformer can efficiently detect spoofing evidence by processing various sequence lengths and aggregating them. In experimental resu

Stats

• citations15
• S2 citations—
• github stars0
• HF likes0
• heat score9.03
• arxiv keyshin2023hm

Related papers

• Heterogeneity Over Homogeneity: Investigating Multilingual Speech Pre-trained Models For Detecting Audio Deepfake (2024)8.09
• Betray Oneself: A Novel Audio Deepfake Detection Model Via Mono-to-stereo Conversion (2023)10.04
• Self-attention And Hybrid Features For Replay And Deep-fake Audio Detection (2024)0.00
• Transsionadd: A Multi-frame Reinforcement Based Sequence Tagging Model For Audio Deepfake Detection (2023)0.00
• Synthetic Voice Detection And Audio Splicing Detection Using Se-res2net-conformer Architecture (2022)6.77
• The Codecfake Dataset And Countermeasures For The Universally Detection Of Deepfake Audio (2024)10.97
• Adversarial Attacks On Audio Deepfake Detection: A Benchmark And Comparative Study (2025)0.00
• What To Remember: Self-adaptive Continual Learning For Audio Deepfake Detection (2023)10.48