Unveiling Deep Semantic Uncertainty Perception For Language-anchored Multi-modal Vision-brain Alignment
2025 · Zehui Feng, Chenqi Zhang, Mingru Wang, et al.
Abstract
Unveiling visual semantics from neural signals such as EEG, MEG, and fMRI remains a fundamental challenge due to subject variability and the entangled nature of visual features. Existing approaches primarily align neural activity directly with visual embeddings, but visual-only representations often fail to capture latent semantic dimensions, limiting interpretability and deep robustness. To address these limitations, we propose Bratrix, the first end-to-end framework to achieve multimodal Language-Anchored Vision-Brain alignment. Bratrix decouples visual stimuli into hierarchical visual and linguistic semantic components, and projects both visual and brain representations into a shared latent space, enabling the formation of aligned visual-language and brain-language embeddings. To emulate human-like perceptual reliability and handle noisy neural signals, Bratrix incorporates a novel uncertainty perception module that applies uncertainty-aware weighting during alignment. By leveraging
Authors
(none)
Tags
Stats
Related papers
- Achieving Fine-grained Cross-modal Understanding Through Brain-inspired Hierarchical Representation Learning (2026)0.00
- Blind To Position, Biased In Language: Probing Mid-layer Representational Bias In Vision-language Encoders For Zero-shot Language-grounded Spatial Understanding (2025)0.00
- Combating Visual Neglect And Semantic Drift In Large Multimodal Models For Enhanced Cross-modal Retrieval (2026)0.00
- Brain-inspired Capture: Evidence-driven Neuromimetic Perceptual Simulation For Visual Decoding (2026)0.00
- Multimodal Representation Alignment For Cross-modal Information Retrieval (2025)0.00
- Learning Robust Visual-semantic Embeddings (2017)15.22
- Human-aligned Image Models Improve Visual Decoding From The Brain (2025)0.00
- X-aligner: Composed Visual Retrieval Without The Bells And Whistles (2026)0.00