Self-supervised Learning With Bi-label Masked Speech Prediction For Streaming Multi-talker Speech Recognition
2022 · Zili Huang, Zhuo Chen, Naoyuki Kanda, et al.
Abstract
Self-supervised learning (SSL), which utilizes the input data itself for representation learning, has achieved state-of-the-art results for various downstream speech tasks. However, most of the previous studies focused on offline single-talker applications, with limited investigations in multi-talker cases, especially for streaming scenarios. In this paper, we investigate SSL for streaming multi-talker speech recognition, which generates transcriptions of overlapping speakers in a streaming fashion. We first observe that conventional SSL techniques do not work well on this task due to the poor representation of overlapping speech. We then propose a novel SSL training objective, referred to as bi-label masked speech prediction, which explicitly preserves representations of all speakers in overlapping speech. We investigate various aspects of the proposed system including data configuration and quantizer selection. The proposed SSL setup achieves substantially better word error rates on
Authors
(none)
Tags
Stats
Related papers
- Downstream Task Agnostic Speech Enhancement With Self-supervised Representation Loss (2023)6.77
- Adapting Self-supervised Models To Multi-talker Speech Recognition Using Speaker Embeddings (2022)10.61
- Multi-resolution Hubert: Multi-resolution Speech Self-supervised Learning With Masked Unit Prediction (2023)0.00
- Deploying Self-supervised Learning In The Wild For Hybrid Automatic Speech Recognition (2022)0.00
- Unispeech-sat: Universal Speech Representation Learning With Speaker Aware Pre-training (2021)0.00
- MT4SSL: Boosting Self-supervised Speech Representation Learning By Integrating Multiple Targets (2022)0.00
- Weakly-supervised Speech Pre-training: A Case Study On Target Speech Recognition (2023)8.09
- Multi-variant Consistency Based Self-supervised Learning For Robust Automatic Speech Recognition (2021)0.00