Speaker Disentanglement Of Speech Pre-trained Model Based On Interpretability
2025 · Xiaoxu Zhu, Junhua Li, Aaron J. Li, et al.
Abstract
Self-supervised speech models learn representations that capture both content and speaker information. Yet this entanglement creates problems: content tasks suffer from speaker bias, and privacy concerns arise when speaker identity leaks through supposedly anonymized representations. We present two contributions to address these challenges. First, we develop InterpTRQE-SptME (Timbre Residual Quantitative Evaluation Benchmark of Speech pre-training Models Encoding via Interpretability), a benchmark that directly measures residual speaker information in content embeddings using SHAP-based interpretability analysis. Unlike existing indirect metrics, our approach quantifies the exact proportion of speaker information remaining after disentanglement. Second, we propose InterpTF-SptME, which uses these interpretability insights to filter speaker information from embeddings. Testing on VCTK with seven models including HuBERT, WavLM, and ContentVec, we find that SHAP Noise filtering reduces sp
Authors
(none)
Tags
Stats
Related papers
- Contentvec: An Improved Self-supervised Speech Representation By Disentangling Speakers (2022)0.00
- Disentangling Voice And Content With Self-supervision For Speaker Recognition (2023)2.26
- Intra-class Variation Reduction Of Speaker Representation In Disentanglement Framework (2020)8.35
- Disentangling Textual And Acoustic Features Of Neural Speech Representations (2024)0.00
- Disentangled-transformer: An Explainable End-to-end Automatic Speech Recognition Model With Speech Content-context Separation (2024)3.58
- Residual Speech Embeddings For Tone Classification: Removing Linguistic Content To Enhance Paralinguistic Analysis (2025)0.00
- Selective Hubert: Self-supervised Pre-training For Target Speaker In Clean And Mixture Speech (2023)7.81
- Self-supervised Disentangled Representation Learning For Robust Target Speech Extraction (2023)5.24