Efficient Adapter Transfer Of Self-supervised Speech Models For Automatic Speech Recognition

Abstract

Self-supervised learning (SSL) is a powerful tool that allows learning of underlying representations from unlabeled data. Transformer based models such as wav2vec 2.0 and HuBERT are leading the field in the speech domain. Generally these models are fine-tuned on a small amount of labeled data for a downstream task such as Automatic Speech Recognition (ASR). This involves re-training the majority of the model for each task. Adapters are small lightweight modules which are commonly used in Natural Language Processing (NLP) to adapt pre-trained models to new tasks. In this paper we propose applying adapters to wav2vec 2.0 to reduce the number of parameters required for downstream ASR tasks, and increase scalability of the model to multiple tasks or languages. Using adapters we can perform ASR while training fewer than 10% of parameters per task compared to full fine-tuning with little degradation of performance. Ablations show that applying adapters into just the top few layers of the pre

Stats

• citations48
• S2 citations—
• github stars0
• HF likes0
• heat score12.68
• arxiv keythomas2022efficient

Related papers

• CHAPTER: Exploiting Convolutional Neural Network Adapters For Self-supervised Speech Models (2022)7.50
• An Adapter Based Pre-training For Efficient And Scalable Self-supervised Speech Representation Learning (2021)8.35
• Efficient Infusion Of Self-supervised Representations In Automatic Speech Recognition (2024)0.00
• Exploring Efficient-tuning Methods In Self-supervised Speech Models (2022)10.07
• Efficient Adapter Tuning Of Pre-trained Speech Models For Automatic Speaker Verification (2024)0.00
• How To Learn A New Language? An Efficient Solution For Self-supervised Learning Models Unseen Languages Adaption In Low-resource Scenario (2024)0.00
• Front-end Adapter: Adapting Front-end Input Of Speech Based Self-supervised Learning For Speech Recognition (2023)0.00
• An Adapter Based Multi-label Pre-training For Speech Separation And Enhancement (2022)7.50