Training Dynamic Models Using Early Exits For Automatic Speech Recognition On Resource-constrained Devices

Abstract

The ability to dynamically adjust the computational load of neural models during inference is crucial for on-device processing scenarios characterised by limited and time-varying computational resources. A promising solution is presented by early-exit architectures, in which additional exit branches are appended to intermediate layers of the encoder. In self-attention models for automatic speech recognition (ASR), early-exit architectures enable the development of dynamic models capable of adapting their size and architecture to varying levels of computational resources and ASR performance demands. Previous research on early-exiting ASR models has relied on pre-trained self-supervised models, fine-tuned with an early-exit loss. In this paper, we undertake an experimental comparison between fine-tuning pre-trained backbones and training models from scratch with the early-exiting objective. Experiments conducted on public datasets reveal that early-exit models trained from scratch not on

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keywright2023training

Related papers

• Dynamic Sparsity Neural Networks For Automatic Speech Recognition (2020)0.00
• Hubert-ee: Early Exiting Hubert For Efficient Speech Recognition (2022)6.77
• Dynamic Encoder Size Based On Data-driven Layer-wise Pruning For Speech Recognition (2024)5.24
• Residual Adapters For Parameter-efficient ASR Adaptation To Atypical And Accented Speech (2021)10.74
• Attention-based ASR With Lightweight And Dynamic Convolutions (2019)9.03
• Gated Low-rank Adaptation For Personalized Code-switching Automatic Speech Recognition On The Low-spec Devices (2024)0.00
• Mobileasr: A Resource-aware On-device Learning Framework For User Voice Personalization Applications On Mobile Phones (2023)0.00
• Bootstrap An End-to-end ASR System By Multilingual Training, Transfer Learning, Text-to-text Mapping And Synthetic Audio (2020)5.24