Data-efficient Targeted Token-level Preference Optimization For Llm-based Text-to-speech

Abstract

arXiv:2510.05799v2 Announce Type: replace-cross Abstract: Aligning text-to-speech (TTS) system outputs with human feedback through preference optimization has been shown to effectively improve the robustness and naturalness of language model-based TTS models. Current approaches primarily require paired desirable and undesirable samples at the utterance level. However, such pairs are often limited in TTS output data, and utterance-level formulation prevents fine-grained token-level optimization needed for accurate pronunciation alignment. In this study, we propose TKTO that eliminates the need for paired data, enabling a more data-efficient training paradigm, and directly targets token-level units, automatically providing fine-grained alignment signals without token-level annotations. TKTO improves the challenging Japanese TTS accuracy by 39% and reduces CER by 54%, automatically assigning 12.8 times stronger reward to targeted tokens.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keykotoge2026data

Related papers

• No Verifiable Reward For Prosody: Toward Preference-guided Prosody Learning In TTS (2025)0.00
• TASTE: Text-aligned Speech Tokenization And Embedding For Spoken Language Modeling (2025)0.00
• Improving Robustness Of Llm-based Speech Synthesis By Learning Monotonic Alignment (2024)0.00
• Non-autoregressive TTS With Explicit Duration Modelling For Low-resource Highly Expressive Speech (2021)8.82
• Emo-dpo: Controllable Emotional Speech Synthesis Through Direct Preference Optimization (2024)9.59
• Efficienttts: An Efficient And High-quality Text-to-speech Architecture (2020)0.00
• Stylespeech: Parameter-efficient Fine Tuning For Pre-trained Controllable Text-to-speech (2024)6.34
• Regotron: Regularizing The Tacotron2 Architecture Via Monotonic Alignment Loss (2022)5.24