Phoneme Based Neural Transducer For Large Vocabulary Speech Recognition

Abstract

To join the advantages of classical and end-to-end approaches for speech recognition, we present a simple, novel and competitive approach for phoneme-based neural transducer modeling. Different alignment label topologies are compared and word-end-based phoneme label augmentation is proposed to improve performance. Utilizing the local dependency of phonemes, we adopt a simplified neural network structure and a straightforward integration with the external word-level language model to preserve the consistency of seq-to-seq modeling. We also present a simple, stable and efficient training procedure using frame-wise cross-entropy loss. A phonetic context size of one is shown to be sufficient for the best performance. A simplified scheduled sampling approach is applied for further improvement and different decoding approaches are briefly compared. The overall performance of our best model is comparable to state-of-the-art (SOTA) results for the TED-LIUM Release 2 and Switchboard corpora.

Stats

• citations18
• S2 citations—
• github stars0
• HF likes0
• heat score9.59
• arxiv keyzhou2020phoneme

Related papers

• Neural Speech Recognizer: Acoustic-to-word LSTM Model For Large Vocabulary Speech Recognition (2016)15.16
• Transformer-transducer: End-to-end Speech Recognition With Self-attention (2019)0.00
• Exploring Neural Transducers For End-to-end Speech Recognition (2017)14.90
• Label-synchronous Neural Transducer For Adaptable Online E2E Speech Recognition (2023)3.58
• Improving RNN Transducer Modeling For End-to-end Speech Recognition (2019)0.00
• A Systematic Comparison Of Grapheme-based Vs. Phoneme-based Label Units For Encoder-decoder-attention Models (2020)0.00
• Knowledge Transfer From Large-scale Pretrained Language Models To End-to-end Speech Recognizers (2022)9.41
• Exploring Pre-training With Alignments For RNN Transducer Based End-to-end Speech Recognition (2020)9.41