A Modulation Module For Multi-task Learning With Applications In Image Retrieval

Abstract

Multi-task learning has been widely adopted in many computer vision tasks to improve overall computation efficiency or boost the performance of individual tasks, under the assumption that those tasks are correlated and complementary to each other. However, the relationships between the tasks are complicated in practice, especially when the number of involved tasks scales up. When two tasks are of weak relevance, they may compete or even distract each other during joint training of shared parameters, and as a consequence undermine the learning of all the tasks. This will raise destructive interference which decreases learning efficiency of shared parameters and lead to low quality loss local optimum w.r.t. shared parameters. To address the this problem, we propose a general modulation module, which can be inserted into any convolutional neural network architecture, to encourage the coupling and feature sharing of relevant tasks while disentangling the learning of irrelevant tasks with m

Stats

• citations87
• S2 citations—
• github stars0
• HF likes0
• heat score14.58
• arxiv keyzhao2018a

Related papers

• Multi-task Cross-modal Learning For Chest X-ray Image Retrieval (2026)0.00
• 12-in-1: Multi-task Vision And Language Representation Learning (2019)17.85
• Brain Tumor Image Retrieval Via Multitask Learning (2018)0.00
• Indexing Multimodal Language Models For Large-scale Image Retrieval (2026)0.00
• CAMP: Cross-modal Adaptive Message Passing For Text-image Retrieval (2019)18.38
• AMC: Attention Guided Multi-modal Correlation Learning For Image Search (2017)10.97
• Multi-modal Reference Learning For Fine-grained Text-to-image Retrieval (2025)6.77
• Learning Unseen Modality Interaction (2023)0.00