Streaming Multi-talker Speech Recognition With Joint Speaker Identification

Abstract

In multi-talker scenarios such as meetings and conversations, speech processing systems are usually required to transcribe the audio as well as identify the speakers for downstream applications. Since overlapped speech is common in this case, conventional approaches usually address this problem in a cascaded fashion that involves speech separation, speech recognition and speaker identification that are trained independently. In this paper, we propose Streaming Unmixing, Recognition and Identification Transducer (SURIT) -- a new framework that deals with this problem in an end-to-end streaming fashion. SURIT employs the recurrent neural network transducer (RNN-T) as the backbone for both speech recognition and speaker identification. We validate our idea on the LibrispeechMix dataset -- a multi-talker dataset derived from Librispeech, and present encouraging results.

Stats

• citations9
• S2 citations—
• github stars0
• HF likes0
• heat score7.50
• arxiv keylu2021streaming

Related papers

• Streaming End-to-end Multi-talker Speech Recognition (2020)11.49
• Continuous Streaming Multi-talker ASR With Dual-path Transducers (2021)7.50
• Streaming Multi-speaker ASR With RNN-T (2020)10.07
• Multi-turn RNN-T For Streaming Recognition Of Multi-party Speech (2021)8.82
• Separator-transducer-segmenter: Streaming Recognition And Segmentation Of Multi-party Speech (2022)0.00
• Joint Speaker Counting, Speech Recognition, And Speaker Identification For Overlapped Speech Of Any Number Of Speakers (2020)12.54
• Hypothesis Clustering And Merging: Novel Multitalker Speech Recognition With Speaker Tokens (2024)0.00
• Integration Of Speech Separation, Diarization, And Recognition For Multi-speaker Meetings: System Description, Comparison, And Analysis (2020)13.23