Self-supervised Predictive Coding Models Encode Speaker And Phonetic Information In Orthogonal Subspaces

Abstract

Self-supervised speech representations are known to encode both speaker and phonetic information, but how they are distributed in the high-dimensional space remains largely unexplored. We hypothesize that they are encoded in orthogonal subspaces, a property that lends itself to simple disentanglement. Applying principal component analysis to representations of two predictive coding models, we identify two subspaces that capture speaker and phonetic variances, and confirm that they are nearly orthogonal. Based on this property, we propose a new speaker normalization method which collapses the subspace that encodes speaker information, without requiring transcriptions. Probing experiments show that our method effectively eliminates speaker information and outperforms a previous baseline in phone discrimination tasks. Moreover, the approach generalizes and can be used to remove information of unseen speakers.

Stats

• citations8
• S2 citations—
• github stars0
• HF likes0
• heat score7.16
• arxiv keyliu2023self

Related papers

• Orthogonality And Isotropy Of Speaker And Phonetic Information In Self-supervised Speech Representations (2024)6.34
• Analyzing Speaker Information In Self-supervised Models To Improve Zero-resource Speech Processing (2021)9.23
• Phone And Speaker Spatial Organization In Self-supervised Speech Representations (2023)2.26
• Contrastive Separative Coding For Self-supervised Representation Learning (2021)0.00
• Neural Predictive Coding Using Convolutional Neural Networks Towards Unsupervised Learning Of Speaker Characteristics (2018)11.85
• Removing Speaker Information From Speech Representation Using Variable-length Soft Pooling (2024)0.00
• Subspace-based Representation And Learning For Phonotactic Spoken Language Recognition (2022)0.00
• Speech Representation Analysis Based On Inter- And Intra-model Similarities (2024)2.26