Articulatory Representation Learning Via Joint Factor Analysis And Neural Matrix Factorization

Abstract

Articulatory representation learning is the fundamental research in modeling neural speech production system. Our previous work has established a deep paradigm to decompose the articulatory kinematics data into gestures, which explicitly model the phonological and linguistic structure encoded with human speech production mechanism, and corresponding gestural scores. We continue with this line of work by raising two concerns: (1) The articulators are entangled together in the original algorithm such that some of the articulators do not leverage effective moving patterns, which limits the interpretability of both gestures and gestural scores; (2) The EMA data is sparsely sampled from articulators, which limits the intelligibility of learned representations. In this work, we propose a novel articulatory representation decomposition algorithm that takes the advantage of guided factor analysis to derive the articulatory-specific factors and factor scores. A neural convolutive matrix factori

Stats

• citations9
• S2 citations—
• github stars0
• HF likes0
• heat score7.50
• arxiv keylian2022articulatory

Related papers

• Deep Neural Convolutive Matrix Factorization For Articulatory Representation Decomposition (2022)7.50
• Learning Robust Speech Representation With An Articulatory-regularized Variational Autoencoder (2021)3.58
• Reconstructing Speech From Real-time Articulatory MRI Using Neural Vocoders (2021)0.00
• A Study Of Incorporating Articulatory Movement Information In Speech Enhancement (2020)0.00
• Simulating Articulatory Trajectories With Phonological Feature Interpolation (2024)0.00
• ARTI-6: Towards Six-dimensional Articulatory Speech Encoding (2025)0.00
• Tracking Articulatory Dynamics In Speech With A Fixed-weight Bilstm-cnn Architecture (2025)0.00
• Feature Joint-state Posterior Estimation In Factorial Speech Processing Models Using Deep Neural Networks (2017)3.58