Memory Time Span In Lstms For Multi-speaker Source Separation

Abstract

With deep learning approaches becoming state-of-the-art in many speech (as well as non-speech) related machine learning tasks, efforts are being taken to delve into the neural networks which are often considered as a black box. In this paper it is analyzed how recurrent neural network (RNNs) cope with temporal dependencies by determining the relevant memory time span in a long short-term memory (LSTM) cell. This is done by leaking the state variable with a controlled lifetime and evaluating the task performance. This technique can be used for any task to estimate the time span the LSTM exploits in that specific scenario. The focus in this paper is on the task of separating speakers from overlapping speech. We discern two effects: A long term effect, probably due to speaker characterization and a short term effect, probably exploiting phone-size formant tracks.

Stats

• citations2
• S2 citations—
• github stars0
• HF likes0
• heat score3.58
• arxiv keyzegers2018memory

Related papers

• Streaming Multi-speaker ASR With RNN-T (2020)10.07
• Dual-path RNN: Efficient Long Sequence Modeling For Time-domain Single-channel Speech Separation (2019)21.06
• Low-latency Deep Clustering For Speech Separation (2019)8.09
• CNN-LSTM Models For Multi-speaker Source Separation Using Bayesian Hyper Parameter Optimization (2019)6.34
• Deep Neural Networks For Automatic Speaker Recognition Do Not Learn Supra-segmental Temporal Features (2023)5.24
• Speaker-conditioned Target Speaker Extraction Based On Customized LSTM Cells (2021)0.00
• Tensor-train Long Short-term Memory For Monaural Speech Enhancement (2018)0.00
• Text-independent Speaker Verification Using Long Short-term Memory Networks (2018)0.00