DRAFT: A Novel Framework To Reduce Domain Shifting In Self-supervised Learning And Its Application To Children's ASR

Abstract

Self-supervised learning (SSL) in the pretraining stage using un-annotated speech data has been successful in low-resource automatic speech recognition (ASR) tasks. However, models trained through SSL are biased to the pretraining data which is usually different from the data used in finetuning tasks, causing a domain shifting problem, and thus resulting in limited knowledge transfer. We propose a novel framework, domain responsible adaptation and finetuning (DRAFT), to reduce domain shifting in pretrained speech models through an additional adaptation stage. In DRAFT, residual adapters (RAs) are inserted in the pretrained model to learn domain-related information with the same SSL loss as the pretraining stage. Only RA parameters are updated during the adaptation stage. DRAFT is agnostic to the type of SSL method used and is evaluated with three widely used approaches: APC, Wav2vec2.0, and HuBERT. On two child ASR tasks (OGI and MyST databases), using SSL models trained with un-annota

Stats

• citations24
• S2 citations—
• github stars0
• HF likes0
• heat score10.48
• arxiv keyfan2022draft

Related papers

• Boosting Cross-domain Speech Recognition With Self-supervision (2022)0.00
• Automatic Data Augmentation For Domain Adapted Fine-tuning Of Self-supervised Speech Representations (2023)0.00
• PADA: Pruning Assisted Domain Adaptation For Self-supervised Speech Representations (2022)5.24
• Deploying Self-supervised Learning In The Wild For Hybrid Automatic Speech Recognition (2022)0.00
• Ac-mix: Self-supervised Adaptation For Low-resource Automatic Speech Recognition Using Agnostic Contrastive Mixup (2024)2.26
• Efficient Adapter Transfer Of Self-supervised Speech Models For Automatic Speech Recognition (2022)12.68
• An Adapter Based Pre-training For Efficient And Scalable Self-supervised Speech Representation Learning (2021)8.35
• How To Learn A New Language? An Efficient Solution For Self-supervised Learning Models Unseen Languages Adaption In Low-resource Scenario (2024)0.00