A Reinforcement Learning Framework For Online Speaker Diarization

Abstract

Speaker diarization is a task to label an audio or video recording with the identity of the speaker at each given time stamp. In this work, we propose a novel machine learning framework to conduct real-time multi-speaker diarization and recognition without prior registration and pretraining in a fully online and reinforcement learning setting. Our framework combines embedding extraction, clustering, and resegmentation into the same problem as an online decision-making problem. We discuss practical considerations and advanced techniques such as the offline reinforcement learning, semi-supervision, and domain adaptation to address the challenges of limited training data and out-of-distribution environments. Our approach considers speaker diarization as a fully online learning problem of the speaker recognition task, where the agent receives no pretraining from any training set before deployment, and learns to detect speaker identity on the fly through reward feedbacks. The paradigm of th

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keylin2023a

Related papers

• Speaker Diarization As A Fully Online Learning Problem In Minivox (2020)0.00
• Overlap-aware Low-latency Online Speaker Diarization Based On End-to-end Local Segmentation (2021)10.35
• Low-latency Online Speaker Diarization With Graph-based Label Generation (2021)8.60
• Sequence-to-sequence Neural Diarization With Automatic Speaker Detection And Representation (2024)6.34
• Interrelate Training And Searching: A Unified Online Clustering Framework For Speaker Diarization (2022)6.77
• Online End-to-end Neural Diarization With Speaker-tracing Buffer (2020)10.74
• Deep Self-supervised Hierarchical Clustering For Speaker Diarization (2020)5.24
• Unsupervised Speaker Diarization In Distributed Iot Networks Using Federated Learning (2024)4.52