Speech Slytherin: Examining The Performance And Efficiency Of Mamba For Speech Separation, Recognition, And Synthesis

Abstract

It is too early to conclude that Mamba is a better alternative to transformers for speech before comparing Mamba with transformers in terms of both performance and efficiency in multiple speech-related tasks. To reach this conclusion, we propose and evaluate three models for three tasks: Mamba-TasNet for speech separation, ConMamba for speech recognition, and VALL-M for speech synthesis. We compare them with transformers of similar sizes in performance, memory, and speed. Our Mamba or Mamba-transformer hybrid models show comparable or higher performance than their transformer counterparts: Sepformer, Conformer, and VALL-E. They are more efficient than transformers in memory and speed for speech longer than a threshold duration, inversely related to the resolution of a speech token. Mamba for separation is the most efficient, and Mamba for recognition is the least. Further, we show that Mamba is not more efficient than transformer for speech shorter than the threshold duration and perfo

Stats

• citations19
• S2 citations—
• github stars114
• HF likes0
• heat score13.88
• arxiv keyjiang2024speech

Related papers

• Dual-path Mamba: Short And Long-term Bidirectional Selective Structured State Space Models For Speech Separation (2024)4.12
• Mamba-seunet: Mamba Unet For Monaural Speech Enhancement (2024)7.16
• An Exploration Of Mamba For Speech Self-supervised Models (2025)1.20
• An Investigation Of Incorporating Mamba For Speech Enhancement (2024)13.70
• Mamba2 Meets Silence: Robust Vocal Source Separation For Sparse Regions (2025)0.00
• Samba-asr: State-of-the-art Speech Recognition Leveraging Structured State-space Models (2025)0.00
• Audio Mamba: Selective State Spaces For Self-supervised Audio Representations (2024)9.23
• Schr\"odinger Bridge Mamba For One-step Speech Enhancement (2025)0.00