Hyperdimensional Cross-modal Alignment Of Frozen Language And Image Models For Efficient Image Captioning

Abstract

Large unimodal foundation models for vision and language encode rich semantic structures, yet aligning them typically requires computationally intensive multimodal fine-tuning. Such approaches depend on large-scale parameter updates, are resource intensive, and can perturb pretrained representations. Emerging evidence suggests, however, that independently trained foundation models may already exhibit latent semantic compatibility, reflecting shared structures in the data they model. This raises a fundamental question: can cross-modal alignment be achieved without modifying the models themselves? Here we introduce HDFLIM (HyperDimensional computing with Frozen Language and Image Models), a framework that establishes cross-modal mappings while keeping pretrained vision and language models fully frozen. HDFLIM projects unimodal embeddings into a shared hyperdimensional space and leverages lightweight symbolic operations -- binding, bundling, and similarity-based retrieval to construct ass

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keydalvi2026hyperdimensional

Related papers

• Generating Images With Multimodal Language Models (2023)6.77
• Linear Alignment Of Vision-language Models For Image Captioning (2023)0.00
• Modest-align: Data-efficient Alignment For Vision-language Models (2025)0.00
• Probvlm: Probabilistic Adapter For Frozen Vision-language Models (2023)13.41
• Multimodal Representation Alignment For Cross-modal Information Retrieval (2025)0.00
• Uclip: Parameter-efficient Multilingual Extension Of Vision-language Models With Unpaired Data (2025)0.00
• LLM2CLIP: Powerful Language Model Unlocks Richer Cross-modality Representation (2024)2.26
• Himo-clip: Modeling Semantic Hierarchy And Monotonicity In Vision-language Alignment (2025)3.01