Deep Generative Factorization For Speech Signal

Abstract

Various information factors are blended in speech signals, which forms the primary difficulty for most speech information processing tasks. An intuitive idea is to factorize speech signal into individual information factors (e.g., phonetic content and speaker trait), though it turns out to be highly challenging. This paper presents a speech factorization approach based on a novel factorial discriminative normalization flow model (factorial DNF). Experiments conducted on a two-factor case that involves phonetic content and speaker trait demonstrates that the proposed factorial DNF has powerful capability to factorize speech signals and outperforms several comparative models in terms of information representation and manipulation.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keysun2020deep

Related papers

• Deep Factorization For Speech Signal (2018)8.82
• On Investigation Of Unsupervised Speech Factorization Based On Normalization Flow (2019)0.00
• Mixture Factorized Auto-encoder For Unsupervised Hierarchical Deep Factorization Of Speech Signal (2019)0.00
• Naturalspeech 3: Zero-shot Speech Synthesis With Factorized Codec And Diffusion Models (2024)0.00
• Content-context Factorized Representations For Automated Speech Recognition (2022)6.34
• Adversarially Learning Disentangled Speech Representations For Robust Multi-factor Voice Conversion (2021)9.92
• Self-supervised Neural Factor Analysis For Disentangling Utterance-level Speech Representations (2023)0.00
• Feature Joint-state Posterior Estimation In Factorial Speech Processing Models Using Deep Neural Networks (2017)3.58