Robust And Unbounded Length Generalization In Autoregressive Transformer-based Text-to-speech

Abstract

Autoregressive (AR) Transformer-based sequence models are known to have difficulty generalizing to sequences longer than those seen during training. When applied to text-to-speech (TTS), these models tend to drop or repeat words or produce erratic output, especially for longer utterances. In this paper, we introduce enhancements aimed at AR Transformer-based encoder-decoder TTS systems that address these robustness and length generalization issues. Our approach uses an alignment mechanism to provide cross-attention operations with relative location information. The associated alignment position is learned as a latent property of the model via backpropagation and requires no external alignment information during training. While the approach is tailored to the monotonic nature of TTS input-output alignment, it is still able to benefit from the flexible modeling power of interleaved multi-head self- and cross-attention operations. A system incorporating these improvements, which we call V

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keybattenberg2024robust

Related papers

• VAENAR-TTS: Variational Auto-encoder Based Non-autoregressive Text-to-speech Synthesis (2021)7.50
• Improving Robustness Of Llm-based Speech Synthesis By Learning Monotonic Alignment (2024)0.00
• Autotts: End-to-end Text-to-speech Synthesis Through Differentiable Duration Modeling (2022)0.00
• S-transformer: Segment-transformer For Robust Neural Speech Synthesis (2020)0.00
• Regotron: Regularizing The Tacotron2 Architecture Via Monotonic Alignment Loss (2022)5.24
• Tts-by-tts: Tts-driven Data Augmentation For Fast And High-quality Speech Synthesis (2020)9.59
• Continuous Autoregressive Modeling With Stochastic Monotonic Alignment For Speech Synthesis (2025)0.00
• TSNAT: Two-step Non-autoregressvie Transformer Models For Speech Recognition (2021)10.61