End-to-end Monaural Multi-speaker ASR System Without Pretraining

Abstract

Recently, end-to-end models have become a popular approach as an alternative to traditional hybrid models in automatic speech recognition (ASR). The multi-speaker speech separation and recognition task is a central task in cocktail party problem. In this paper, we present a state-of-the-art monaural multi-speaker end-to-end automatic speech recognition model. In contrast to previous studies on the monaural multi-speaker speech recognition, this end-to-end framework is trained to recognize multiple label sequences completely from scratch. The system only requires the speech mixture and corresponding label sequences, without needing any indeterminate supervisions obtained from non-mixture speech or corresponding labels/alignments. Moreover, we exploited using the individual attention module for each separated speaker and the scheduled sampling to further improve the performance. Finally, we evaluate the proposed model on the 2-speaker mixed speech generated from the WSJ corpus and the ws

Stats

• citations38
• S2 citations—
• github stars0
• HF likes0
• heat score11.93
• arxiv keychang2018end

Related papers

• A Purely End-to-end System For Multi-speaker Speech Recognition (2018)12.25
• Survey Of End-to-end Multi-speaker Automatic Speech Recognition For Monaural Audio (2025)2.26
• Investigation Of End-to-end Speaker-attributed ASR For Continuous Multi-talker Recordings (2020)10.35
• A Comparative Study Of Modular And Joint Approaches For Speaker-attributed ASR On Monaural Long-form Audio (2021)7.50
• MIMO-SPEECH: End-to-end Multi-channel Multi-speaker Speech Recognition (2019)13.93
• Elevating Robust Multi-talker ASR By Decoupling Speaker Separation And Speech Recognition (2025)0.00
• End-to-end Multichannel Speaker-attributed ASR: Speaker Guided Decoder And Input Feature Analysis (2023)0.00
• Streaming Multi-speaker ASR With RNN-T (2020)10.07