MMLU

Dataset Card for MMLU Dataset Summary Measuring Massive Multitask Language Understanding by Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt (ICLR 2021). This is a massive multitask test consisting of multiple-choice questions from various branches of knowledge. The

Papers using MMLU (82)

HRM-Text: Efficient Pretraining Beyond Scaling2026The Price of Format: Diversity Collapse in LLMs2025 · 2 citesA general tensor-structured compression scheme for efficient large language models2026Indic-TunedLens: Interpreting Multilingual Models in Indian Languages2026Same Meaning, Different Scores: Lexical and Syntactic Sensitivity in LLM Evaluation2026Confidence Geometry Reveals Trace-Level Correctness in Large Language Model Reasoning2026DataFlex: A Unified Framework for Data-Centric Dynamic Training of Large Language Models2026Locas: Your Models are Principled Initializers of Locally-Supported Parametric Memories2026Lie to Me: How Faithful Is Chain-of-Thought Reasoning in Reasoning Models?2026Benchmarking EngGPT2-16B-A3B against Comparable Italian and International Open-source LLMs2026AtManRL: Towards Faithful Reasoning via Differentiable Attention Saliency2026Learning When to Sample: Confidence-Aware Self-Consistency for Efficient LLM Chain-of-Thought Reasoning2026Residual Stream Analysis of Overfitting And Structural Disruptions2026SCAN: Sparse Circuit Anchor Interpretable Neuron for Lifelong Knowledge Editing2026How do LLMs Compute Verbal Confidence2026FIT: Defying Catastrophic Forgetting in Continual LLM Unlearning2026Leveraging KV Similarity for Online Structured Pruning in LLMs2025Uncertainty-Aware Answer Selection for Improved Reasoning in Multi-LLM Systems2025Fine-Tuning on Noisy Instructions: Effects on Generalization and Performance2025Dr.LLM: Dynamic Layer Routing in LLMs2025Beyond Majority Voting: LLM Aggregation by Leveraging Higher-Order Information2025EAPO: Enhancing Policy Optimization with On-Demand Expert Assistance2025On Robustness and Reliability of Benchmark-Based Evaluation of LLMs2025CorrSteer: Generation-Time LLM Steering via Correlated Sparse Autoencoder Features2025Thinking Inside the Mask: In-Place Prompting in Diffusion LLMs2025Turning the Spell Around: Lightweight Alignment Amplification via Rank-One Safety Injection2025How to Train a Leader: Hierarchical Reasoning in Multi-Agent LLMs2025Uncovering Cross-Linguistic Disparities in LLMs using Sparse Autoencoders2025GEM: Empowering LLM for both Embedding Generation and Language Understanding2025Token Constraint Decoding Improves Robustness on Question Answering for Large Language Models2025From Threat to Tool: Leveraging Refusal-Aware Injection Attacks for Safety Alignment2025TPTT: Transforming Pretrained Transformers into Titans2025SELT: Self-Evaluation Tree Search for LLMs with Task Decomposition2025Control LLM: Controlled Evolution for Intelligence Retention in LLM2025Humanity's Last Exam2025LM2: Large Memory Models2025Selective Self-to-Supervised Fine-Tuning for Generalization in Large Language Models2025LLM-Microscope: Uncovering the Hidden Role of Punctuation in Context Memory of Transformers2025YourBench: Easy Custom Evaluation Sets for Everyone2025AttentionInfluence: Adopting Attention Head Influence for Weak-to-Strong Pretraining Data Selection2025Void in Language Models2025B-score: Detecting biases in large language models using response history2025Interleaved Reasoning for Large Language Models via Reinforcement Learning2025ProtoReasoning: Prototypes as the Foundation for Generalizable Reasoning in LLMs2025TPTT: Transforming Pretrained Transformer into Titans2025Growing Transformers: Modular Composition and Layer-wise Expansion on a Frozen Substrate2025Lizard: An Efficient Linearization Framework for Large Language Models2025CorrSteer: Steering Improves Task Performance and Safety in LLMs through Correlation-based Sparse Autoencoder Feature Selection2025Turning the Spell Around: Lightweight Alignment Amplification via Rank-One Safety Injection2025Fine-Tuning on Noisy Instructions: Effects on Generalization and Performance2025A Single Character can Make or Break Your LLM Evals2025The Atomic Instruction Gap: Instruction-Tuned LLMs Struggle with Simple, Self-Contained Directives2025Parrot: Persuasion and Agreement Robustness Rating of Output Truth -- A Sycophancy Robustness Benchmark for LLMs2025Say It Another Way: Auditing LLMs with a User-Grounded Automated Paraphrasing Framework2025A Scaling Law for Token Efficiency in LLM Fine-Tuning Under Fixed Compute Budgets2025AttentionInfluence: Adopting Attention Head Influence for Weak-to-Strong Pretraining Data Selection2025Decentralized Arena: Towards Democratic and Scalable Automatic Evaluation of Language Models2025Dual Decomposition of Weights and Singular Value Low Rank Adaptation2025Calibrating LLM Confidence by Probing Perturbed Representation Stability2025Self-Reasoning Language Models: Unfold Hidden Reasoning Chains with Few Reasoning Catalyst2025Efficient Data Selection at Scale via Influence Distillation2025More is Less: The Pitfalls of Multi-Model Synthetic Preference Data in DPO Safety Alignment2025Large Language Models Could Be Rote Learners2025None of the Above, Less of the Right: Parallel Patterns between Humans and LLMs on Multi-Choice Questions Answering2025Order Independence With Finetuning2025Shifting Perspectives: Steering Vectors for Robust Bias Mitigation in LLMs2025Rethinking Mixture-of-Agents: Is Mixing Different Large Language Models Beneficial?2025FRAME: Boosting LLMs with A Four-Quadrant Multi-Stage Pretraining Strategy2025Forget What You Know about LLMs Evaluations -- LLMs are Like a Chameleon2025Leveraging Uncertainty Estimation for Efficient LLM Routing2025Towards Fully Exploiting LLM Internal States to Enhance Knowledge Boundary Perception2025Evaluating Expert Contributions in a MoE LLM for Quiz-Based Tasks2025ORI: O Routing Intelligence2025Preference Curriculum: LLMs Should Always Be Pretrained on Their Preferred Data2025The Vulnerability of Language Model Benchmarks: Do They Accurately Reflect True LLM Performance?2024 · 2 citesBridging the Gap: Enhancing LLM Performance for Low-Resource African Languages with New Benchmarks, Fine-Tuning, and Cultural Adjustments2024 · 2 citesLoLCATs: On Low-Rank Linearizing of Large Language Models2024 · 1 citesBenTo: Benchmark Task Reduction with In-Context Transferability2024Adaptive Segment-level Reward: Bridging the Gap Between Action and Reward Space in Alignment2024TODO: Enhancing LLM Alignment with Ternary Preferences2024Reasoning Robustness of LLMs to Adversarial Typographical Errors2024Llama 3 Meets MoE: Efficient Upcycling2024