Open RL Benchmark: Comprehensive Tracked Experiments For Reinforcement Learning

Abstract

In many Reinforcement Learning (RL) papers, learning curves are useful indicators to measure the effectiveness of RL algorithms. However, the complete raw data of the learning curves are rarely available. As a result, it is usually necessary to reproduce the experiments from scratch, which can be time-consuming and error-prone. We present Open RL Benchmark, a set of fully tracked RL experiments, including not only the usual data such as episodic return, but also all algorithm-specific and system metrics. Open RL Benchmark is community-driven: anyone can download, use, and contribute to the data. At the time of writing, more than 25,000 runs have been tracked, for a cumulative duration of more than 8 years. Open RL Benchmark covers a wide range of RL libraries and reference implementations. Special care is taken to ensure that each experiment is precisely reproducible by providing not only the full parameters, but also the versions of the dependencies used to generate it. In addition, O

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyhuang2024open

Related papers

• Reliable Validation Of Reinforcement Learning Benchmarks (2022)0.00
• RL Unplugged: A Suite Of Benchmarks For Offline Reinforcement Learning (2020)0.00
• Xrl-bench: A Benchmark For Evaluating And Comparing Explainable Reinforcement Learning Techniques (2024)0.00
• SLM Lab: A Comprehensive Benchmark And Modular Software Framework For Reproducible Deep Reinforcement Learning (2019)0.00
• Neorl-2: Near Real-world Benchmarks For Offline Reinforcement Learning With Extended Realistic Scenarios (2025)0.00
• D4RL: Datasets For Deep Data-driven Reinforcement Learning (2020)0.00
• A Comprehensive Survey Of Reinforcement Learning: From Algorithms To Practical Challenges (2024)0.00
• Toybox: A Suite Of Environments For Experimental Evaluation Of Deep Reinforcement Learning (2019)0.00