Cross-modal Speaker Verification And Recognition: A Multilingual Perspective

Abstract

Recent years have seen a surge in finding association between faces and voices within a cross-modal biometric application along with speaker recognition. Inspired from this, we introduce a challenging task in establishing association between faces and voices across multiple languages spoken by the same set of persons. The aim of this paper is to answer two closely related questions: "Is face-voice association language independent?" and "Can a speaker be recognised irrespective of the spoken language?". These two questions are very important to understand effectiveness and to boost development of multilingual biometric systems. To answer them, we collected a Multilingual Audio-Visual dataset, containing human speech clips of \(154\) identities with \(3\) language annotations extracted from various videos uploaded online. Extensive experiments on the three splits of the proposed dataset have been performed to investigate and answer these novel research questions that clearly point out th

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keysaeed2020cross

Related papers

• Cross-modal Audio-visual Co-learning For Text-independent Speaker Verification (2023)9.23
• Contrastive Learning-based Chaining-cluster For Multilingual Voice-face Association (2024)4.78
• Multi-stage Face-voice Association Learning With Keynote Speaker Diarization (2024)7.97
• Comparative Analysis Of Modality Fusion Approaches For Audio-visual Person Identification And Verification (2024)0.00
• Face-voice Association In Multilingual Environments (FAME) Challenge 2024 Evaluation Plan (2024)0.00
• A Text-independent Speaker Verification Model: A Comparative Analysis (2017)8.60
• Audio-visual Approach For Multimodal Concurrent Speaker Detection (2024)0.00
• Multi-domain Adaptation By Self-supervised Learning For Speaker Verification (2023)0.00