Efficient Transformer-based Speech Enhancement Using Long Frames And STFT Magnitudes

Abstract

The SepFormer architecture shows very good results in speech separation. Like other learned-encoder models, it uses short frames, as they have been shown to obtain better performance in these cases. This results in a large number of frames at the input, which is problematic; since the SepFormer is transformer-based, its computational complexity drastically increases with longer sequences. In this paper, we employ the SepFormer in a speech enhancement task and show that by replacing the learned-encoder features with a magnitude short-time Fourier transform (STFT) representation, we can use long frames without compromising perceptual enhancement performance. We obtained equivalent quality and intelligibility evaluation scores while reducing the number of operations by a factor of approximately 8 for a 10-second utterance.

Stats

• citations18
• S2 citations—
• github stars0
• HF likes0
• heat score9.59
• arxiv keydeoliveira2022efficient

Related papers

• Consep: A Noise- And Reverberation-robust Speech Separation Framework By Magnitude Conditioning (2024)0.00
• Speechformer: A Hierarchical Efficient Framework Incorporating The Characteristics Of Speech (2022)12.99
• Speechformer++: A Hierarchical Efficient Framework For Paralinguistic Speech Processing (2023)14.43
• Time-domain Speech Enhancement Assisted By Multi-resolution Frequency Encoder And Decoder (2023)9.76
• Resource-efficient Separation Transformer (2022)7.81
• Attention Is All You Need In Speech Separation (2020)20.59
• Dasformer: Deep Alternating Spectrogram Transformer For Multi/single-channel Speech Separation (2023)0.00
• Tiny-sepformer: A Tiny Time-domain Transformer Network For Speech Separation (2022)8.82