Re-examination Of The Role Of Latent Variables In Sequence Modeling

Abstract

With latent variables, stochastic recurrent models have achieved state-of-the-art performance in modeling sound-wave sequence. However, opposite results are also observed in other domains, where standard recurrent networks often outperform stochastic models. To better understand this discrepancy, we re-examine the roles of latent variables in stochastic recurrent models for speech density estimation. Our analysis reveals that under the restriction of fully factorized output distribution in previous evaluations, the stochastic models were implicitly leveraging intra-step correlation but the standard recurrent baselines were prohibited to do so, resulting in an unfair comparison. To correct the unfairness, we remove such restriction in our re-examination, where all the models can explicitly leverage intra-step correlation with an auto-regressive structure. Over a diverse set of sequential data, including human speech, MIDI music, handwriting trajectory and frame-permuted speech, our resu

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keydai2019re

Related papers

• Regularized Sequential Latent Variable Models With Adversarial Neural Networks (2021)2.26
• Sequential Neural Models With Stochastic Layers (2016)0.00
• Stochastic Wavenet: A Generative Latent Variable Model For Sequential Data (2018)0.00
• Rethinking Recurrent Latent Variable Model For Music Composition (2018)7.50
• Benchmarking Generative Latent Variable Models For Speech (2022)0.00
• An Empirical Evaluation Of Generic Convolutional And Recurrent Networks For Sequence Modeling (2018)7.31
• Disentangling Speech And Non-speech Components For Building Robust Acoustic Models From Found Data (2019)0.00
• Investigating Gated Recurrent Neural Networks For Speech Synthesis (2016)0.00