Resource-efficient Separation Transformer

Abstract

Transformers have recently achieved state-of-the-art performance in speech separation. These models, however, are computationally demanding and require a lot of learnable parameters. This paper explores Transformer-based speech separation with a reduced computational cost. Our main contribution is the development of the Resource-Efficient Separation Transformer (RE-SepFormer), a self-attention-based architecture that reduces the computational burden in two ways. First, it uses non-overlapping blocks in the latent space. Second, it operates on compact latent summaries calculated from each chunk. The RE-SepFormer reaches a competitive performance on the popular WSJ0-2Mix and WHAM! datasets in both causal and non-causal settings. Remarkably, it scales significantly better than the previous Transformer-based architectures in terms of memory and inference time, making it more suitable for processing long mixtures.

Stats

• citations10
• S2 citations—
• github stars0
• HF likes0
• heat score7.81
• arxiv keydellalibera2022resource

Related papers

• Attention Is All You Need In Speech Separation (2020)20.59
• Exploring Self-attention Mechanisms For Speech Separation (2022)12.54
• Tiny-sepformer: A Tiny Time-domain Transformer Network For Speech Separation (2022)8.82
• Transmask: A Compact And Fast Speech Separation Model Based On Transformer (2021)8.82
• Monaural Multi-speaker Speech Separation Using Efficient Transformer Model (2023)0.00
• Multi-dimensional And Multi-scale Modeling For Speech Separation Optimized By Discriminative Learning (2023)0.00
• Conv-transformer Transducer: Low Latency, Low Frame Rate, Streamable End-to-end Speech Recognition (2020)11.08
• Efficientasr: Speech Recognition Network Compression Via Attention Redundancy And Chunk-level FFN Optimization (2024)3.58