Location-relative Attention Mechanisms For Robust Long-form Speech Synthesis

Abstract

Despite the ability to produce human-level speech for in-domain text, attention-based end-to-end text-to-speech (TTS) systems suffer from text alignment failures that increase in frequency for out-of-domain text. We show that these failures can be addressed using simple location-relative attention mechanisms that do away with content-based query/key comparisons. We compare two families of attention mechanisms: location-relative GMM-based mechanisms and additive energy-based mechanisms. We suggest simple modifications to GMM-based attention that allow it to align quickly and consistently during training, and introduce a new location-relative attention mechanism to the additive energy-based family, called Dynamic Convolution Attention (DCA). We compare the various mechanisms in terms of alignment speed and consistency during training, naturalness, and ability to generalize to long utterances, and conclude that GMM attention and DCA can generalize to very long utterances, while preserving

Stats

• citations55
• S2 citations—
• github stars0
• HF likes0
• heat score13.11
• arxiv keybattenberg2019location

Related papers

• Improving Robustness Of Llm-based Speech Synthesis By Learning Monotonic Alignment (2024)0.00
• Generating Synthetic Audio Data For Attention-based Speech Recognition Systems (2019)12.68
• An Online Attention-based Model For Speech Recognition (2018)9.59
• Initial Investigation Of An Encoder-decoder End-to-end TTS Framework Using Marginalization Of Monotonic Hard Latent Alignments (2019)0.00
• Transformer-based End-to-end Speech Recognition With Residual Gaussian-based Self-attention (2021)5.84
• Aligner-guided Training Paradigm: Advancing Text-to-speech Models With Aligner Guided Duration (2024)0.00
• Small-e: Small Language Model With Linear Attention For Efficient Speech Synthesis (2024)9.02
• Transformer-based End-to-end Speech Recognition With Local Dense Synthesizer Attention (2020)12.04