A Hitchhiker's Guide To Statistical Comparisons Of Reinforcement Learning Algorithms

Abstract

Consistently checking the statistical significance of experimental results is the first mandatory step towards reproducible science. This paper presents a hitchhiker's guide to rigorous comparisons of reinforcement learning algorithms. After introducing the concepts of statistical testing, we review the relevant statistical tests and compare them empirically in terms of false positive rate and statistical power as a function of the sample size (number of seeds) and effect size. We further investigate the robustness of these tests to violations of the most common hypotheses (normal distributions, same distributions, equal variances). Beside simulations, we compare empirical distributions obtained by running Soft-Actor Critic and Twin-Delayed Deep Deterministic Policy Gradient on Half-Cheetah. We conclude by providing guidelines and code to perform rigorous comparisons of RL algorithm performances.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keycolas2019a

Related papers

• Measuring The Reliability Of Reinforcement Learning Algorithms (2019)4.43
• Deep Reinforcement Learning At The Edge Of The Statistical Precipice (2021)0.00
• Reproducibility Of Benchmarked Deep Reinforcement Learning Tasks For Continuous Control (2017)0.00
• Reliable Validation Of Reinforcement Learning Benchmarks (2022)0.00
• Performance Comparisons Of Reinforcement Learning Algorithms For Sequential Experimental Design (2025)0.00
• A Comprehensive Survey Of Reinforcement Learning: From Algorithms To Practical Challenges (2024)0.00
• Assessing The Impact Of Distribution Shift On Reinforcement Learning Performance (2024)0.00
• Beyond Expected Return: Accounting For Policy Reproducibility When Evaluating Reinforcement Learning Algorithms (2023)3.58