Contentvec: An Improved Self-supervised Speech Representation By Disentangling Speakers

Abstract

Self-supervised learning in speech involves training a speech representation network on a large-scale unannotated speech corpus, and then applying the learned representations to downstream tasks. Since the majority of the downstream tasks of SSL learning in speech largely focus on the content information in speech, the most desirable speech representations should be able to disentangle unwanted variations, such as speaker variations, from the content. However, disentangling speakers is very challenging, because removing the speaker information could easily result in a loss of content as well, and the damage of the latter usually far outweighs the benefit of the former. In this paper, we propose a new SSL method that can achieve speaker disentanglement without severe loss of content. Our approach is adapted from the HuBERT framework, and incorporates disentangling mechanisms to regularize both the teacher labels and the learned representations. We evaluate the benefit of speaker disenta

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyqian2022contentvec

Related papers

• Disentangling Voice And Content With Self-supervision For Speaker Recognition (2023)2.26
• Unispeech-sat: Universal Speech Representation Learning With Speaker Aware Pre-training (2021)0.00
• Self-supervised Disentangled Representation Learning For Robust Target Speech Extraction (2023)5.24
• Towards The Next Frontier In Speech Representation Learning Using Disentanglement (2024)0.00
• Intra-class Variation Reduction Of Speaker Representation In Disentanglement Framework (2020)8.35
• Self-supervised Neural Factor Analysis For Disentangling Utterance-level Speech Representations (2023)0.00
• Speaker Disentanglement Of Speech Pre-trained Model Based On Interpretability (2025)0.00
• Pushing The Limits Of Unsupervised Unit Discovery For SSL Speech Representation (2023)6.34