Content-based Search For Deep Generative Models

Abstract

The growing proliferation of customized and pretrained generative models has made it infeasible for a user to be fully cognizant of every model in existence. To address this need, we introduce the task of content-based model search: given a query and a large set of generative models, finding the models that best match the query. As each generative model produces a distribution of images, we formulate the search task as an optimization problem to select the model with the highest probability of generating similar content as the query. We introduce a formulation to approximate this probability given the query from different modalities, e.g., image, sketch, and text. Furthermore, we propose a contrastive learning framework for model retrieval, which learns to adapt features for various query modalities. We demonstrate that our method outperforms several baselines on Generative Model Zoo, a new benchmark we create for the model retrieval task.

Stats

• citations6
• S2 citations—
• github stars0
• HF likes0
• heat score6.34
• arxiv keylu2022content

Related papers

• Irgen: Generative Modeling For Image Retrieval (2023)7.16
• Look, Imagine And Match: Improving Textual-visual Cross-modal Retrieval With Generative Models (2017)18.52
• Generative Retrieval As Dense Retrieval (2023)0.00
• Generative Retrieval As Multi-vector Dense Retrieval (2024)8.60
• Generative Model For Zero-shot Sketch-based Image Retrieval (2019)9.23
• Chatsearch: A Dataset And A Generative Retrieval Model For General Conversational Image Retrieval (2024)2.00
• Advancing Similarity Search With Genai: A Retrieval Augmented Generation Approach (2024)0.00
• Cross-modal RAG: Sub-dimensional Text-to-image Retrieval-augmented Generation (2025)0.00