RL Unplugged: A Suite Of Benchmarks For Offline Reinforcement Learning

Abstract

Offline methods for reinforcement learning have a potential to help bridge the gap between reinforcement learning research and real-world applications. They make it possible to learn policies from offline datasets, thus overcoming concerns associated with online data collection in the real-world, including cost, safety, or ethical concerns. In this paper, we propose a benchmark called RL Unplugged to evaluate and compare offline RL methods. RL Unplugged includes data from a diverse range of domains including games (e.g., Atari benchmark) and simulated motor control problems (e.g., DM Control Suite). The datasets include domains that are partially or fully observable, use continuous or discrete actions, and have stochastic vs. deterministic dynamics. We propose detailed evaluation protocols for each domain in RL Unplugged and provide an extensive analysis of supervised learning and offline RL methods using these protocols. We will release data for all our tasks and open-source all algor

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keygulcehre2020rl

Related papers

• D4RL: Datasets For Deep Data-driven Reinforcement Learning (2020)0.00
• Neorl-2: Near Real-world Benchmarks For Offline Reinforcement Learning With Extended Realistic Scenarios (2025)0.00
• Bridging The Gap Between Offline And Online Reinforcement Learning Evaluation Methodologies (2022)0.00
• AD4RL: Autonomous Driving Benchmarks For Offline Reinforcement Learning With Value-based Dataset (2024)7.16
• An Optimistic Perspective On Offline Reinforcement Learning (2019)0.00
• AWAC: Accelerating Online Reinforcement Learning With Offline Datasets (2020)0.00
• Ogbench: Benchmarking Offline Goal-conditioned RL (2024)0.00
• Expert-supervised Reinforcement Learning For Offline Policy Learning And Evaluation (2020)0.00