Meta-learning Empowered Meta-face: Personalized Speaking Style Adaptation For Audio-driven 3D Talking Face Animation

Abstract

Audio-driven 3D face animation is increasingly vital in live streaming and augmented reality applications. While remarkable progress has been observed, most existing approaches are designed for specific individuals with predefined speaking styles, thus neglecting the adaptability to varied speaking styles. To address this limitation, this paper introduces MetaFace, a novel methodology meticulously crafted for speaking style adaptation. Grounded in the novel concept of meta-learning, MetaFace is composed of several key components: the Robust Meta Initialization Stage (RMIS) for fundamental speaking style adaptation, the Dynamic Relation Mining Neural Process (DRMN) for forging connections between observed and unobserved speaking styles, and the Low-rank Matrix Memory Reduction Approach to enhance the efficiency of model optimization as well as learning style details. Leveraging these novel designs, MetaFace not only significantly outperforms robust existing baselines but also establishe

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhou2024meta

Related papers

• Meta-tts: Meta-learning For Few-shot Speaker Adaptive Text-to-speech (2021)12.74
• Facespeak: Expressive And High-quality Speech Synthesis From Human Portraits Of Different Styles (2025)0.00
• Diffusiontalker: Efficient And Compact Speech-driven 3D Talking Head Via Personalizer-guided Distillation (2025)5.05
• Meta-voice: Fast Few-shot Style Transfer For Expressive Voice Cloning Using Meta Learning (2021)0.00
• Pmmtalk: Speech-driven 3D Facial Animation From Complementary Pseudo Multi-modal Features (2023)3.58
• Sample Efficient Adaptive Text-to-speech (2018)0.00
• Learning To Adapt: A Meta-learning Approach For Speaker Adaptation (2018)9.76
• The Universal Personalizer: Few-shot Dysarthric Speech Recognition Via Meta-learning (2025)0.00