Learning To Hash-tag Videos With Tag2vec

Abstract

User-given tags or labels are valuable resources for semantic understanding of visual media such as images and videos. Recently, a new type of labeling mechanism known as hash-tags have become increasingly popular on social media sites. In this paper, we study the problem of generating relevant and useful hash-tags for short video clips. Traditional data-driven approaches for tag enrichment and recommendation use direct visual similarity for label transfer and propagation. We attempt to learn a direct low-cost mapping from video to hash-tags using a two step training process. We first employ a natural language processing (NLP) technique, skip-gram models with neural network training to learn a low-dimensional vector representation of hash-tags (Tag2Vec) using a corpus of 10 million hash-tags. We then train an embedding function to map video features to the low-dimensional Tag2vec space. We learn this embedding for 29 categories of short video clips with hash-tags. A query video without

Stats

• citations2
• S2 citations—
• github stars0
• HF likes0
• heat score3.58
• arxiv keysingh2016learning

Related papers

• Weakly Supervised Deep Image Hashing Through Tag Embeddings (2018)11.29
• CHAIN: Exploring Global-local Spatio-temporal Information For Improved Self-supervised Video Hashing (2023)8.60
• Doctag2vec: An Embedding Based Multi-label Learning Approach For Document Tagging (2017)8.60
• Video Retrieval Based On Deep Convolutional Neural Network (2017)9.03
• Self-supervised Video Hashing With Hierarchical Binary Auto-encoder (2018)17.81
• Tagging Before Alignment: Integrating Multi-modal Tags For Video-text Retrieval (2023)10.74
• Encode The Unseen: Predictive Video Hashing For Scalable Mid-stream Retrieval (2020)3.58
• A Feature-space Multimodal Data Augmentation Technique For Text-video Retrieval (2022)12.43