MMstar

MMStar (Are We on the Right Way for Evaluating Large Vision-Language Models?) 🌐 Homepage | 🤗 Dataset | 🤗 Paper | 📖 arXiv | GitHub Dataset Details As shown in the figure below, existing benchmarks lack consideration of the vision dependency of evaluation samples and potential data leakage from LLMs' and LVLMs' train

Papers using MMstar (2)

UPME: An Unsupervised Peer Review Framework for Multimodal Large Language Model Evaluation2025Open-Qwen2VL: Compute-Efficient Pre-Training of Fully-Open Multimodal LLMs on Academic Resources2025