UltraFeedback

Introduction GitHub Repo UltraRM-13b UltraCM-13b UltraFeedback is a large-scale, fine-grained, diverse preference dataset, used for training powerful reward models and critic models. We collect about 64k prompts from diverse resources (including UltraChat, ShareGPT, Evol-Instruct, TruthfulQA, FalseQA, and FLAN). We the

Papers using UltraFeedback (5)

When Data is the Algorithm: A Systematic Study and Curation of Preference Optimization Datasets2025Adaptive Margin RLHF via Preference over Preferences2025ShiQ: Bringing back Bellman to LLMs2025Provably Efficient Online RLHF with One-Pass Reward Modeling2025Self-Evolved Reward Learning for LLMs2024 · 1 cites