Flowavse: Efficient Audio-visual Speech Enhancement With Conditional Flow Matching

Abstract

This work proposes an efficient method to enhance the quality of corrupted speech signals by leveraging both acoustic and visual cues. While existing diffusion-based approaches have demonstrated remarkable quality, their applicability is limited by slow inference speeds and computational complexity. To address this issue, we present FlowAVSE which enhances the inference speed and reduces the number of learnable parameters without degrading the output quality. In particular, we employ a conditional flow matching algorithm that enables the generation of high-quality speech in a single sampling step. Moreover, we increase efficiency by optimizing the underlying U-net architecture of diffusion-based systems. Our experiments demonstrate that FlowAVSE achieves 22 times faster inference speed and reduces the model size by half while maintaining the output quality. The demo page is available at: https://cyongong.github.io/FlowAVSE.github.io/

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyjung2024flowavse

Related papers

• Voiceflow: Efficient Text-to-speech With Rectified Flow Matching (2023)0.00
• Flashaudio: Rectified Flows For Fast And High-fidelity Text-to-audio Generation (2024)5.13
• Flowhigh: Towards Efficient And High-quality Audio Super-resolution With Single-step Flow Matching (2025)5.84
• Av2wav: Diffusion-based Re-synthesis From Continuous Self-supervised Features For Audio-visual Speech Enhancement (2023)0.00
• Voicerestore: Flow-matching Transformers For Speech Recording Quality Restoration (2025)0.00
• Real-time Streamable Generative Speech Restoration With Flow Matching (2025)0.00
• Meanflowse: One-step Generative Speech Enhancement Via Conditional Mean Flow (2025)3.01
• Diffusion-based Unsupervised Audio-visual Speech Enhancement (2024)4.52