Token-level Ensemble Distillation For Grapheme-to-phoneme Conversion
2019 · Hao Sun, Xu Tan, Jun-Wei Gan, et al.
Abstract
Grapheme-to-phoneme (G2P) conversion is an important task in automatic speech recognition and text-to-speech systems. Recently, G2P conversion is viewed as a sequence to sequence task and modeled by RNN or CNN based encoder-decoder framework. However, previous works do not consider the practical issues when deploying G2P model in the production system, such as how to leverage additional unlabeled data to boost the accuracy, as well as reduce model size for online deployment. In this work, we propose token-level ensemble distillation for G2P conversion, which can (1) boost the accuracy by distilling the knowledge from additional unlabeled data, and (2) reduce the model size but maintain the high accuracy, both of which are very practical and helpful in the online production system. We use token-level knowledge distillation, which results in better accuracy than the sequence-level counterpart. What is more, we adopt the Transformer instead of RNN or CNN based models to further boost the
Authors
(none)
Tags
Stats
Related papers
- One Model To Pronounce Them All: Multilingual Grapheme-to-phoneme Conversion With A Transformer Ensemble (2020)0.00
- Massively Multilingual Neural Grapheme-to-phoneme Conversion (2017)9.76
- Liteg2p: A Fast, Light And High Accuracy Model For Grapheme-to-phoneme Conversion (2023)5.84
- R-g2p: Evaluating And Enhancing Robustness Of Grapheme To Phoneme Conversion By Controlled Noise Introducing And Contextual Information Incorporation (2022)7.50
- Transformer Based Grapheme-to-phoneme Conversion (2020)11.39
- Data-driven Grapheme-to-phoneme Representations For A Lexicon-free Text-to-speech (2024)4.52
- Improving Grapheme-to-phoneme Conversion Through In-context Knowledge Retrieval With Large Language Models (2024)2.26
- Codert: Distilling Encoder Representations With Co-learning For Transducer-based Speech Recognition (2021)6.77