Revisiting Self-supervised Learning Of Speech Representation From A Mutual Information Perspective

Abstract

Existing studies on self-supervised speech representation learning have focused on developing new training methods and applying pre-trained models for different applications. However, the quality of these models is often measured by the performance of different downstream tasks. How well the representations access the information of interest is less studied. In this work, we take a closer look into existing self-supervised methods of speech from an information-theoretic perspective. We aim to develop metrics using mutual information to help practical problems such as model design and selection. We use linear probes to estimate the mutual information between the target information and learned representations, showing another insight into the accessibility to the target information from speech representations. Further, we explore the potential of evaluating representations in a self-supervised fashion, where we estimate the mutual information between different parts of the data without u

Stats

• citations3
• S2 citations—
• github stars0
• HF likes0
• heat score4.52
• arxiv keyliu2024revisiting

Related papers

• Similarity Analysis Of Self-supervised Speech Representations (2020)10.07
• Layer-wise Analysis Of A Self-supervised Speech Representation Model (2021)17.07
• Learning Speaker Representations With Mutual Information (2018)11.76
• Orthogonality And Isotropy Of Speaker And Phonetic Information In Self-supervised Speech Representations (2024)6.34
• Learning Problem-agnostic Speech Representations From Multiple Self-supervised Tasks (2019)15.54
• Speech Representation Analysis Based On Inter- And Intra-model Similarities (2024)2.26
• Perceive And Predict: Self-supervised Speech Representation Based Loss Functions For Speech Enhancement (2023)7.16
• An Exploration Of Self-supervised Pretrained Representations For End-to-end Speech Recognition (2021)12.25