Investigation Of Speaker Representation For Target-speaker Speech Processing

Abstract

Target-speaker speech processing (TS) tasks, such as target-speaker automatic speech recognition (TS-ASR), target speech extraction (TSE), and personal voice activity detection (p-VAD), are important for extracting information about a desired speaker's speech even when it is corrupted by interfering speakers. While most studies have focused on training schemes or system architectures for each specific task, the auxiliary network for embedding target-speaker cues has not been investigated comprehensively in a unified cross-task evaluation. Therefore, this paper aims to address a fundamental question: what is the preferred speaker embedding for TS tasks? To this end, for the TS-ASR, TSE, and p-VAD tasks, we compare pre-trained speaker encoders (i.e., self-supervised or speaker recognition models) that compute speaker embeddings from pre-recorded enrollment speech of the target speaker with ideal speaker embeddings derived directly from the target speaker's identity in the form of a one-h

Stats

• citations3
• S2 citations—
• github stars0
• HF likes0
• heat score4.52
• arxiv keyashihara2024investigation

Related papers

• An Analysis On The Effects Of Speaker Embedding Choice In Non Auto-regressive TTS (2023)0.00
• Quantitative Evidence On Overlooked Aspects Of Enrollment Speaker Embeddings For Target Speaker Separation (2022)7.16
• Investigation Of Speaker-adaptation Methods In Transformer Based ASR (2020)0.00
• Learning Speaker Embedding From Text-to-speech (2020)5.84
• S-vectors And TESA: Speaker Embeddings And A Speaker Authenticator Based On Transformer Encoder (2020)11.08
• New Insights On Target Speaker Extraction (2022)0.00
• Adapting Self-supervised Models To Multi-talker Speech Recognition Using Speaker Embeddings (2022)10.61
• USEF-TSE: Universal Speaker Embedding Free Target Speaker Extraction (2024)11.88