Auxiliary Loss Of Transformer With Residual Connection For End-to-end Speaker Diarization

Abstract

End-to-end neural diarization (EEND) with self-attention directly predicts speaker labels from inputs and enables the handling of overlapped speech. Although the EEND outperforms clustering-based speaker diarization (SD), it cannot be further improved by simply increasing the number of encoder blocks because the last encoder block is dominantly supervised compared with lower blocks. This paper proposes a new residual auxiliary EEND (RX-EEND) learning architecture for transformers to enforce the lower encoder blocks to learn more accurately. The auxiliary loss is applied to the output of each encoder block, including the last encoder block. The effect of auxiliary loss on the learning of the encoder blocks can be further increased by adding a residual connection between the encoder blocks of the EEND. Performance evaluation and ablation study reveal that the auxiliary loss in the proposed RX-EEND provides relative reductions in the diarization error rate (DER) by 50.3% and 21.0% on the

Stats

• citations13
• S2 citations—
• github stars0
• HF likes0
• heat score8.60
• arxiv keyyu2021auxiliary

Related papers

• Improving Transformer-based End-to-end Speaker Diarization By Assigning Auxiliary Losses To Attention Heads (2023)7.16
• Speech-aware Neural Diarization With Encoder-decoder Attractor Guided By Attention Constraints (2024)0.00
• Transformer Attractors For Robust And Efficient End-to-end Neural Diarization (2023)6.77
• End-to-end Neural Diarization: From Transformer To Conformer (2021)10.85
• Towards Word-level End-to-end Neural Speaker Diarization With Auxiliary Network (2023)0.00
• EEND-DEMUX: End-to-end Neural Speaker Diarization Via Demultiplexed Speaker Embeddings (2023)0.00
• Encoder-decoder Based Attractors For End-to-end Neural Diarization (2021)13.05
• Speakers Unembedded: Embedding-free Approach To Long-form Neural Diarization (2024)3.58