ADAPTERMIX: Exploring The Efficacy Of Mixture Of Adapters For Low-resource TTS Adaptation

Abstract

There are significant challenges for speaker adaptation in text-to-speech for languages that are not widely spoken or for speakers with accents or dialects that are not well-represented in the training data. To address this issue, we propose the use of the "mixture of adapters" method. This approach involves adding multiple adapters within a backbone-model layer to learn the unique characteristics of different speakers. Our approach outperforms the baseline, with a noticeable improvement of 5% observed in speaker preference tests when using only one minute of data for each new speaker. Moreover, following the adapter paradigm, we fine-tune only the adapter parameters (11% of the total model parameters). This is a significant achievement in parameter-efficient speaker adaptation, and one of the first models of its kind. Overall, our proposed approach offers a promising solution to the speech synthesis techniques, particularly for adapting to speakers from diverse backgrounds.

Stats

• citations6
• S2 citations—
• github stars0
• HF likes0
• heat score6.34
• arxiv keymehrish2023adaptermix

Related papers

• Adapter-based Extension Of Multi-speaker Text-to-speech Model For New Speakers (2022)6.77
• Voicetailor: Lightweight Plug-in Adapter For Diffusion-based Personalized Text-to-speech (2024)3.58
• Residual Adapters For Parameter-efficient ASR Adaptation To Atypical And Accented Speech (2021)10.74
• Efficient Adapter Tuning Of Pre-trained Speech Models For Automatic Speaker Verification (2024)0.00
• SLM-TTA: A Framework For Test-time Adaptation Of Generative Spoken Language Models (2025)0.00
• Elp-adapters: Parameter Efficient Adapter Tuning For Various Speech Processing Tasks (2024)7.81
• Hypertts: Parameter Efficient Adaptation In Text To Speech Using Hypernetworks (2024)3.23
• Sample Efficient Adaptive Text-to-speech (2018)0.00