Nanovoice: Efficient Speaker-adaptive Text-to-speech For Multiple Speakers

Abstract

We present NanoVoice, a personalized text-to-speech model that efficiently constructs voice adapters for multiple speakers simultaneously. NanoVoice introduces a batch-wise speaker adaptation technique capable of fine-tuning multiple references in parallel, significantly reducing training time. Beyond building separate adapters for each speaker, we also propose a parameter sharing technique that reduces the number of parameters used for speaker adaptation. By incorporating a novel trainable scale matrix, NanoVoice mitigates potential performance degradation during parameter sharing. NanoVoice achieves performance comparable to the baselines, while training 4 times faster and using 45 percent fewer parameters for speaker adaptation with 40 reference voices. Extensive ablation studies and analysis further validate the efficiency of our model.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keypark2024nanovoice

Related papers

• Adapter-based Extension Of Multi-speaker Text-to-speech Model For New Speakers (2022)6.77
• Sample Efficient Adaptive Text-to-speech (2018)0.00
• Voicetailor: Lightweight Plug-in Adapter For Diffusion-based Personalized Text-to-speech (2024)3.58
• ADAPTERMIX: Exploring The Efficacy Of Mixture Of Adapters For Low-resource TTS Adaptation (2023)6.34
• Speaker-adaptive Neural Vocoders For Parametric Speech Synthesis Systems (2018)2.26
• Nnspeech: Speaker-guided Conditional Variational Autoencoder For Zero-shot Multi-speaker Text-to-speech (2022)9.59
• Deep Voice 2: Multi-speaker Neural Text-to-speech (2017)0.00
• Hypertts: Parameter Efficient Adaptation In Text To Speech Using Hypernetworks (2024)3.23