TSTNN: Two-stage Transformer Based Neural Network For Speech Enhancement In The Time Domain

Abstract

In this paper, we propose a transformer-based architecture, called two-stage transformer neural network (TSTNN) for end-to-end speech denoising in the time domain. The proposed model is composed of an encoder, a two-stage transformer module (TSTM), a masking module and a decoder. The encoder maps input noisy speech into feature representation. The TSTM exploits four stacked two-stage transformer blocks to efficiently extract local and global information from the encoder output stage by stage. The masking module creates a mask which will be multiplied with the encoder output. Finally, the decoder uses the masked encoder feature to reconstruct the enhanced speech. Experimental results on the benchmark dataset show that the TSTNN outperforms most state-of-the-art models in time or frequency domain while having significantly lower model complexity.

Stats

• citations169
• S2 citations—
• github stars0
• HF likes0
• heat score16.73
• arxiv keywang2021tstnn

Related papers

• Invertible Dnn-based Nonlinear Time-frequency Transform For Speech Enhancement (2019)7.16
• Dense-tsnet: Dense Connected Two-stage Structure For Ultra-lightweight Speech Enhancement (2024)0.00
• Single Channel Speech Enhancement Using Temporal Convolutional Recurrent Neural Networks (2020)5.84
• Study Of Lightweight Transformer Architectures For Single-channel Speech Enhancement (2025)3.58
• TFCN: Temporal-frequential Convolutional Network For Single-channel Speech Enhancement (2022)0.00
• TSNAT: Two-step Non-autoregressvie Transformer Models For Speech Recognition (2021)10.61
• Tensor-to-vector Regression For Multi-channel Speech Enhancement Based On Tensor-train Network (2020)12.39
• S-transformer: Segment-transformer For Robust Neural Speech Synthesis (2020)0.00