Less Than Few: Self-shot Video Instance Segmentation

Abstract

The goal of this paper is to bypass the need for labelled examples in few-shot video understanding at run time. While proven effective, in many practical video settings even labelling a few examples appears unrealistic. This is especially true as the level of details in spatio-temporal video understanding and with it, the complexity of annotations continues to increase. Rather than performing few-shot learning with a human oracle to provide a few densely labelled support videos, we propose to automatically learn to find appropriate support videos given a query. We call this self-shot learning and we outline a simple self-supervised learning method to generate an embedding space well-suited for unsupervised retrieval of relevant samples. To showcase this novel setting, we tackle, for the first time, video instance segmentation in a self-shot (and few-shot) setting, where the goal is to segment instances at the pixel-level across the spatial and temporal domains. We provide strong baseli

Stats

• citations1
• S2 citations—
• github stars0
• HF likes0
• heat score2.26
• arxiv keyyang2022less

Related papers

• Learnable Prompt For Few-shot Semantic Segmentation In Remote Sensing Domain (2024)7.16
• Blazingly Fast Video Object Segmentation With Pixel-wise Metric Learning (2018)17.46
• Few-shot Learning Through An Information Retrieval Lens (2017)0.00
• Multimodal Clustering Networks For Self-supervised Learning From Unlabeled Videos (2021)13.28
• Viseret: A Simple Yet Effective Approach To Moment Retrieval Via Fine-grained Video Segmentation (2021)0.00
• Shotfinder: Imagination-driven Open-domain Video Shot Retrieval Via Web Search (2026)0.00
• Finding Significant Features For Few-shot Learning Using Dimensionality Reduction (2021)2.26
• CHAIN: Exploring Global-local Spatio-temporal Information For Improved Self-supervised Video Hashing (2023)8.60