GPQA

Dataset Card for GPQA GPQA is a multiple-choice, Q&A dataset of very hard questions written and validated by experts in biology, physics, and chemistry. When attempting questions out of their own domain (e.g., a physicist answers a chemistry question), these experts get only 34% accuracy, despite spending >30m with ful

Papers using GPQA (32)

The Evolution of Thought: Tracking LLM Overthinking via Reasoning Dynamics Analysis2025 · 10 citesRethinking LLM-as-a-Judge: Representation-as-a-Judge with Small Language Models via Semantic Capacity Asymmetry2026Evaluating and Mitigating LLM-as-a-judge Bias in Communication Systems2025 · 3 citesDecision Protocols in Multi-Agent Large Language Model Conversations2026When LLMs Agree, Are They Right? Auditing Self-Consistency and Cross-Model Agreement as Confidence Signals2026Data Repetition Beats Data Scaling in Long-CoT Supervised Fine-Tuning2026FocuSFT: Bilevel Optimization for Dilution-Aware Long-Context Fine-Tuning2026NGM: A Plug-and-Play Training-Free Memory Module for LLMs2026Apriel-Reasoner: RL Post-Training for General-Purpose and Efficient Reasoning2026SePO: Self-Evolving Prompt Agent for System Prompt Optimization2026TERMINATOR: Learning Optimal Exit Points for Early Stopping in Chain-of-Thought Reasoning2026Benchmark Illusion: Disagreement Among Llms And Its Scientific Consequences2026Budget-Aware Anytime Reasoning with LLM-Synthesized Preference Data2026Dr.LLM: Dynamic Layer Routing in LLMs2025AutoBench: Automating LLM Evaluation through Reciprocal Peer Assessment2025EAPO: Enhancing Policy Optimization with On-Demand Expert Assistance2025CAC-CoT: Connector-Aware Compact Chain-of-Thought for Efficient Reasoning Data Synthesis Across Dual-System Cognitive Tasks2025P3: Prompts Promote Prompting2025Efficient Model Development through Fine-tuning Transfer2025General-Reasoner: Advancing LLM Reasoning Across All Domains2025Interleaved Reasoning for Large Language Models via Reinforcement Learning2025Enigmata: Scaling Logical Reasoning in Large Language Models with Synthetic Verifiable Puzzles2025Inference-Time Hyper-Scaling with KV Cache Compression2025Fractional Reasoning via Latent Steering Vectors Improves Inference Time Compute2025DeepPrune: Parallel Scaling without Inter-trace Redundancy2025MCTS-RAG: Enhancing Retrieval-Augmented Generation with Monte Carlo Tree Search2025Reinforcing General Reasoning without Verifiers2025From Harm to Help: Turning Reasoning In-Context Demos into Assets for Reasoning LMs2025Reasoning with Sampling: Your Base Model is Smarter Than You Think2025Can LLMs Guide Their Own Exploration? Gradient-Guided Reinforcement Learning for LLM Reasoning2025INFERENCEDYNAMICS: Efficient Routing Across LLMs through Structured Capability and Knowledge Profiling2025Dancing with Critiques: Enhancing LLM Reasoning with Stepwise Natural Language Self-Critique2025