PADA: Pruning Assisted Domain Adaptation For Self-supervised Speech Representations
2022 · Lodagala V S V Durga Prasad, Sreyan Ghosh, S. Umesh
Abstract
While self-supervised speech representation learning (SSL) models serve a variety of downstream tasks, these models have been observed to overfit to the domain from which the unlabelled data originates. To alleviate this issue, we propose PADA (Pruning Assisted Domain Adaptation) and zero out redundant weights from models pre-trained on large amounts of out-of-domain (OOD) data. Intuitively, this helps to make space for the target-domain ASR finetuning. The redundant weights can be identified through various pruning strategies which have been discussed in detail as a part of this work. Specifically, we investigate the effect of the recently discovered Task-Agnostic and Task-Aware pruning on PADA and propose a new pruning paradigm based on the latter, which we call Cross-Domain Task-Aware Pruning (CD-TAW). CD-TAW obtains the initial pruning mask from a well fine-tuned OOD model, which makes it starkly different from the rest of the pruning strategies discussed in the paper. Our proposed
Authors
(none)
Tags
Stats
Related papers
- Automatic Data Augmentation For Domain Adapted Fine-tuning Of Self-supervised Speech Representations (2023)0.00
- Boosting Cross-domain Speech Recognition With Self-supervision (2022)0.00
- DRAFT: A Novel Framework To Reduce Domain Shifting In Self-supervised Learning And Its Application To Children's ASR (2022)10.48
- Synergistic Effects Of Knowledge Distillation And Structured Pruning For Self-supervised Speech Models (2025)0.00
- Structured Pruning Of Self-supervised Pre-trained Models For Speech Recognition And Understanding (2023)11.39
- Self-supervised Learning Based Domain Adaptation For Robust Speaker Verification (2021)11.49
- A Domain Adaptation Framework For Speech Recognition Systems With Only Synthetic Data (2025)5.24
- Unsupervised Domain Adaptation For Speech Recognition Via Uncertainty Driven Self-training (2020)12.25