Gated Low-rank Adaptation For Personalized Code-switching Automatic Speech Recognition On The Low-spec Devices

Abstract

In recent times, there has been a growing interest in utilizing personalized large models on low-spec devices, such as mobile and CPU-only devices. However, utilizing a personalized large model in the on-device is inefficient, and sometimes limited due to computational cost. To tackle the problem, this paper presents the weights separation method to minimize on-device model weights using parameter-efficient fine-tuning methods. Moreover, some people speak multiple languages in an utterance, as known as code-switching, the personalized ASR model is necessary to address such cases. However, current multilingual speech recognition models are limited to recognizing a single language within each utterance. To tackle this problem, we propose code-switching speech recognition models that incorporate fine-tuned monolingual and multilingual speech recognition models. Additionally, we introduce a gated low-rank adaptation(GLoRA) for parameter-efficient fine-tuning with minimal performance degrad

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keykim2024gated

Related papers

• Residual Adapters For Parameter-efficient ASR Adaptation To Atypical And Accented Speech (2021)10.74
• Language Modeling For Code-switching: Evaluation, Integration Of Monolingual Data, And Discriminative Training (2018)5.24
• Adaptive Activation Network For Low Resource Multilingual Speech Recognition (2022)0.00
• Fast Contextual Adaptation With Neural Associative Memory For On-device Personalized Speech Recognition (2021)9.76
• Mobileasr: A Resource-aware On-device Learning Framework For User Voice Personalization Applications On Mobile Phones (2023)0.00
• Investigating Training Strategies And Model Robustness Of Low-rank Adaptation For Language Modeling In Speech Recognition (2024)0.00
• Generative Error Correction For Code-switching Speech Recognition Using Large Language Models (2023)0.00
• Parameter-efficient Adaptation Of Multilingual Multimodal Models For Low-resource ASR (2024)2.26