UltraFeedback

Introduction GitHub Repo UltraRM-13b UltraCM-13b UltraFeedback is a large-scale, fine-grained, diverse preference dataset, used for training powerful reward models and critic models. We collect about 64k prompts from diverse resources (including UltraChat, ShareGPT, Evol-Instruct, TruthfulQA, FalseQA, and FLAN). We the

Papers using UltraFeedback (6)

Less is More: Improving LLM Alignment via Preference Data Selection2025 · 1 citesMGDA-Decoupled: Geometry-Aware Multi-Objective Optimisation for DPO-based LLM Alignment2026Intelligently Weighting Multiple Reference Models for Direct Preference Optimization of LLMs2025When Data is the Algorithm: A Systematic Study and Curation of Preference Optimization Datasets2025Beyond Majority Voting: LLM Aggregation by Leveraging Higher-Order Information2025CoPL: Collaborative Preference Learning for Personalizing LLMs2025