Learning Speech Representations With Variational Predictive Coding
2025 · Sung-Lin Yeh, Peter Bell, Hao Tang
Abstract
Despite being the best known objective for learning speech representations, the HuBERT objective has not been further developed and improved. We argue that it is the lack of an underlying principle that stalls the development, and, in this paper, we show that predictive coding under a variational view is the principle behind the HuBERT objective. Due to its generality, our formulation provides opportunities to improve parameterization and optimization, and we show two simple modifications that bring immediate improvements to the HuBERT objective. In addition, the predictive coding formulation has tight connections to various other objectives, such as APC, CPC, wav2vec, and BEST-RQ. Empirically, the improvement in pre-training brings significant improvements to four downstream tasks: phone classification, f0 tracking, speaker recognition, and automatic speech recognition, highlighting the importance of the predictive coding interpretation.
Authors
(none)
Tags
Stats
Related papers
- Improved Speech Representations With Multi-target Autoregressive Predictive Coding (2020)10.97
- Analysing The Masked Predictive Coding Training Criterion For Pre-training A Speech Representation Model (2023)4.52
- Self-supervised Representation Learning With Relative Predictive Coding (2021)0.00
- Ms-hubert: Mitigating Pre-training And Inference Mismatch In Masked Language Modelling Methods For Learning Speech Representations (2024)4.52
- Cobert: Self-supervised Speech Representation Learning Through Code Representation Learning (2022)3.38
- Neural Feature Predictor And Discriminative Residual Coding For Low-bitrate Speech Coding (2022)6.77
- Speech Representation Learning Revisited: The Necessity Of Separate Learnable Parameters And Robust Data Augmentation (2024)0.00
- Disentangled Speech Representation Learning Based On Factorized Hierarchical Variational Autoencoder With Self-supervised Objective (2022)7.81