D4RL: Datasets For Deep Data-driven Reinforcement Learning

Abstract

The offline reinforcement learning (RL) setting (also known as full batch RL), where a policy is learned from a static dataset, is compelling as progress enables RL methods to take advantage of large, previously-collected datasets, much like how the rise of large datasets has fueled results in supervised learning. However, existing online RL benchmarks are not tailored towards the offline setting and existing offline RL benchmarks are restricted to data generated by partially-trained agents, making progress in offline RL difficult to measure. In this work, we introduce benchmarks specifically designed for the offline setting, guided by key properties of datasets relevant to real-world applications of offline RL. With a focus on dataset collection, examples of such properties include: datasets generated via hand-designed controllers and human demonstrators, multitask datasets where an agent performs different tasks in the same environment, and datasets collected with mixtures of policie

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyfu2020d4rl

Related papers

• AD4RL: Autonomous Driving Benchmarks For Offline Reinforcement Learning With Value-based Dataset (2024)7.16
• Data Valuation For Offline Reinforcement Learning (2022)0.00
• RL Unplugged: A Suite Of Benchmarks For Offline Reinforcement Learning (2020)0.00
• An Optimistic Perspective On Offline Reinforcement Learning (2019)0.00
• A Dataset Perspective On Offline Reinforcement Learning (2021)0.00
• AWAC: Accelerating Online Reinforcement Learning With Offline Datasets (2020)0.00
• Neorl-2: Near Real-world Benchmarks For Offline Reinforcement Learning With Extended Realistic Scenarios (2025)0.00
• D3rlpy: An Offline Deep Reinforcement Learning Library (2021)0.00