Measuring The Reliability Of Reinforcement Learning Algorithms

Abstract

Lack of reliability is a well-known issue for reinforcement learning (RL) algorithms. This problem has gained increasing attention in recent years, and efforts to improve it have grown substantially. To aid RL researchers and production users with the evaluation and improvement of reliability, we propose a set of metrics that quantitatively measure different aspects of reliability. In this work, we focus on variability and risk, both during training and after learning (on a fixed policy). We designed these metrics to be general-purpose, and we also designed complementary statistical tests to enable rigorous comparisons on these metrics. In this paper, we first describe the desired properties of the metrics and their design, the aspects of reliability that they measure, and their applicability to different scenarios. We then describe the statistical tests and make additional practical recommendations for reporting results. The metrics and accompanying statistical tools have been made av

Stats

• citations0
• S2 citations—
• github stars164
• HF likes0
• heat score4.43
• arxiv keychan2019measuring

Related papers

• Reliable Validation Of Reinforcement Learning Benchmarks (2022)0.00
• Beyond Expected Return: Accounting For Policy Reproducibility When Evaluating Reinforcement Learning Algorithms (2023)3.58
• Assessing The Impact Of Distribution Shift On Reinforcement Learning Performance (2024)0.00
• Review Of Metrics To Measure The Stability, Robustness And Resilience Of Reinforcement Learning (2022)0.00
• Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, And Generalizability (2022)0.00
• A Hitchhiker's Guide To Statistical Comparisons Of Reinforcement Learning Algorithms (2019)0.00
• Deep Reinforcement Learning At The Edge Of The Statistical Precipice (2021)0.00
• Ablation Study Of How Run Time Assurance Impacts The Training And Performance Of Reinforcement Learning Agents (2022)6.77