Multi-dialect Speech Recognition With A Single Sequence-to-sequence Model

Abstract

Sequence-to-sequence models provide a simple and elegant solution for building speech recognition systems by folding separate components of a typical system, namely acoustic (AM), pronunciation (PM) and language (LM) models into a single neural network. In this work, we look at one such sequence-to-sequence model, namely listen, attend and spell (LAS), and explore the possibility of training a single model to serve different English dialects, which simplifies the process of training multi-dialect systems without the need for separate AM, PM and LMs for each dialect. We show that simply pooling the data from all dialects into one LAS model falls behind the performance of a model fine-tuned on each dialect. We then look at incorporating dialect-specific information into the model, both by modifying the training targets by inserting the dialect symbol at the end of the original grapheme sequence and also feeding a 1-hot representation of the dialect information into all layers of the mode

Stats

• citations68
• S2 citations—
• github stars0
• HF likes0
• heat score13.79
• arxiv keyli2017multi

Related papers

• State-of-the-art Speech Recognition With Sequence-to-sequence Models (2017)21.01
• Multilingual Speech Recognition With A Single End-to-end Model (2017)16.05
• A Highly Adaptive Acoustic Model For Accurate Multi-dialect Speech Recognition (2022)10.85
• Multilingual Sequence-to-sequence Speech Recognition: Architecture, Transfer Learning, And Language Modeling (2018)13.84
• Towards One Model To Rule All: Multilingual Strategy For Dialectal Code-switching Arabic ASR (2021)9.03
• A Purely End-to-end System For Multi-speaker Speech Recognition (2018)12.25
• Analysis Of Multilingual Sequence-to-sequence Speech Recognition Systems (2018)0.00
• Towards Relevance And Sequence Modeling In Language Recognition (2020)9.23