Graph-pit: Generalized Permutation Invariant Training For Continuous Separation Of Arbitrary Numbers Of Speakers

Abstract

Automatic transcription of meetings requires handling of overlapped speech, which calls for continuous speech separation (CSS) systems. The uPIT criterion was proposed for utterance-level separation with neural networks and introduces the constraint that the total number of speakers must not exceed the number of output channels. When processing meeting-like data in a segment-wise manner, i.e., by separating overlapping segments independently and stitching adjacent segments to continuous output streams, this constraint has to be fulfilled for any segment. In this contribution, we show that this constraint can be significantly relaxed. We propose a novel graph-based PIT criterion, which casts the assignment of utterances to output channels in a graph coloring problem. It only requires that the number of concurrently active speakers must not exceed the number of output channels. As a consequence, the system can process an arbitrary number of speakers and arbitrarily long segments and thus

Stats

• citations14
• S2 citations—
• github stars0
• HF likes0
• heat score8.82
• arxiv keyvonneumann2021graph

Related papers

• Separating Long-form Speech With Group-wise Permutation Invariant Training (2021)4.52
• Probabilistic Permutation Invariant Training For Speech Separation (2019)7.81
• Permutation Invariant Training Of Deep Models For Speaker-independent Multi-talker Speech Separation (2016)0.00
• Multi-talker Speech Separation With Utterance-level Permutation Invariant Training Of Deep Recurrent Neural Networks (2017)20.90
• Interrupted And Cascaded Permutation Invariant Training For Speech Separation (2019)4.52
• Single-channel Speech Separation Using Soft-minimum Permutation Invariant Training (2021)2.26
• Transcription-free Fine-tuning Of Speech Separation Models For Noisy And Reverberant Multi-speaker Automatic Speech Recognition (2024)3.58
• Utterance-level Permutation Invariant Training With Latency-controlled BLSTM For Single-channel Multi-talker Speech Separation (2019)0.00