SL-DML: Signal Level Deep Metric Learning For Multimodal One-shot Action Recognition

Abstract

Recognizing an activity with a single reference sample using metric learning approaches is a promising research field. The majority of few-shot methods focus on object recognition or face-identification. We propose a metric learning approach to reduce the action recognition problem to a nearest neighbor search in embedding space. We encode signals into images and extract features using a deep residual CNN. Using triplet loss, we learn a feature embedding. The resulting encoder transforms features into an embedding space in which closer distances encode similar actions while higher distances encode different actions. Our approach is based on a signal level formulation and remains flexible across a variety of modalities. It further outperforms the baseline on the large scale NTU RGB+D 120 dataset for the One-Shot action recognition protocol by 5.6%. With just 60% of the training data, our approach still outperforms the baseline approach by 3.7%. With 40% of the training data, our approac

Stats

• citations31
• S2 citations—
• github stars0
• HF likes0
• heat score11.29
• arxiv keymemmesheimer2020sl

Related papers

• Skeleton-dml: Deep Metric Learning For Skeleton-based One-shot Action Recognition (2020)12.17
• Multi-level Similarity Learning For Low-shot Recognition (2019)0.00
• Guided Deep Metric Learning (2022)6.77
• Few-shot Metric Learning: Online Adaptation Of Embedding For Retrieval (2022)8.09
• Human Motion Analysis With Deep Metric Learning (2018)11.58
• Indirect: Language-guided Zero-shot Deep Metric Learning For Images (2022)5.24
• Hybrid-attention Based Decoupled Metric Learning For Zero-shot Image Retrieval (2019)12.93
• Revisiting Metric Learning For Few-shot Image Classification (2019)14.90