MMLU-Pro

MMLU-Pro Dataset MMLU-Pro dataset is a more robust and challenging massive multi-task understanding dataset tailored to more rigorously benchmark large language models' capabilities. This dataset contains 12K complex questions across various disciplines. |Github | 🏆Leaderboard | 📖Paper | 🚀 What's New [2026.03.11] Ad

Papers using MMLU-Pro (29)

K-Quantization and its Impact on Output Performance2026NITP: Next Implicit Token Prediction for LLM Pre-training2026D-COT: Disciplined Chain-of-Thought Learning for Efficient Reasoning in Small Language Models2026Benchmark Illusion: Disagreement Among Llms And Its Scientific Consequences2026AWPO: Enhancing Tool-Use of Large Language Models through Adaptive Integration of Reasoning Rewards2025Group-Aware Reinforcement Learning for Output Diversity in Large Language Models2025Let the Model Distribute Its Doubt: Confidence Estimation through Verbalized Probability Distribution2025AutoBench: Automating LLM Evaluation through Reciprocal Peer Assessment2025Can Multiple Responses from an LLM Reveal the Sources of Its Uncertainty?2025Persuasion Dynamics in LLMs: Investigating Robustness and Adaptability in Knowledge and Safety with DuET-PD2025Token Constraint Decoding Improves Robustness on Question Answering for Large Language Models2025OctoTools: An Agentic Framework with Extensible Tools for Complex Reasoning2025When an LLM is apprehensive about its answers -- and when its uncertainty is justified2025T1: Tool-integrated Self-verification for Test-time Compute Scaling in Small Language Models2025AttentionInfluence: Adopting Attention Head Influence for Weak-to-Strong Pretraining Data Selection2025Warm Up Before You Train: Unlocking General Reasoning in Resource-Constrained Settings2025General-Reasoner: Advancing LLM Reasoning Across All Domains2025Reinforcing General Reasoning without Verifiers2025DyePack: Provably Flagging Test Set Contamination in LLMs Using Backdoors2025Answer Matching Outperforms Multiple Choice for Language Model Evaluation2025From KMMLU-Redux to KMMLU-Pro: A Professional Korean Benchmark Suite for LLM Evaluation2025Persuasion Dynamics in LLMs: Investigating Robustness and Adaptability in Knowledge and Safety with DuET-PD2025The Atomic Instruction Gap: Instruction-Tuned LLMs Struggle with Simple, Self-Contained Directives2025Can LLMs Guide Their Own Exploration? Gradient-Guided Reinforcement Learning for LLM Reasoning2025AttentionInfluence: Adopting Attention Head Influence for Weak-to-Strong Pretraining Data Selection2025ReviewInstruct: A Review-Driven Multi-Turn Conversations Generation Method for Large Language Models2025INFERENCEDYNAMICS: Efficient Routing Across LLMs through Structured Capability and Knowledge Profiling2025Calibrating LLM Confidence by Probing Perturbed Representation Stability2025Distill Not Only Data but Also Rewards: Can Smaller Language Models Surpass Larger Ones?2025