MBPP

Mostly Basic Python Problems — ~1,000 entry-level programming tasks with tests, for evaluating code generation.

Papers using MBPP (26)

Can Language Models Replace Programmers For Coding? REPOCOD Says 'not Yet'2024 · 3 citesHeteroskedastic Signals in Budgeted LLM Verification: Structural Heterogeneity Limits Optimization Gains2026OpenCodeInstruct: A Large-scale Instruction Tuning Dataset for Code LLMs2025 · 1 citesACE: Self-Evolving LLM Coding Framework via Adversarial Unit Test Generation and Preference Optimization2026Fast-dLLM++: Fr\'{e}chet Profile Decoding for Faster Diffusion LLM Inference2026Signature filtering: a lightweight enhancement for statistical watermark detection in large language models2026Reward-Free Code Alignment from Pretrained or Fine-Tuned LLM: Unpacking the Trade-offs for Code Generation2026EpiCaR: Knowing What You Don't Know Matters for Better Reasoning in LLMs2026Locally Confident, Globally Stuck: The Quality-Exploration Dilemma in Diffusion Language Models2026Can I Have Your Order? Monte-Carlo Tree Search for Slot Filling Ordering in Diffusion Language Models2026SePO: Self-Evolving Prompt Agent for System Prompt Optimization2026DMax: Aggressive Parallel Decoding for dLLMs2026Guided Collaboration in Heterogeneous LLM-Based Multi-Agent Systems via Entropy-Based Understanding Assessment and Experience Retrieval2026TGPR: Tree-Guided Policy Refinement for Robust Self-Debugging of LLMs2025From Implicit Exploration to Structured Reasoning: Leveraging Guideline and Refinement for LLMs2025Efficient Code LLM Training via Distribution-Consistent and Diversity-Aware Data Selection2025Enhancing Reasoning Capabilities of Small Language Models with Blueprints and Prompt Template Search2025Shape of Thought: When Distribution Matters More than Correctness in Reasoning Tasks2025Learning to Insert [PAUSE] Tokens for Better Reasoning2025Learning to Generate Unit Tests for Automated Debugging2025CodeCriticBench: A Holistic Code Critique Benchmark for Large Language Models2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Models2025dParallel: Learnable Parallel Decoding for dLLMs2025Aligning CodeLLMs with Direct Preference Optimization2024 · 1 citesSelf-Explained Keywords Empower Large Language Models for Code Generation2024Learning to Reason via Self-Iterative Process Feedback for Small Language Models2024