Provably Optimal Reinforcement Learning Under Safety Filtering

Abstract

Recent advances in reinforcement learning (RL) enable its use on increasingly complex tasks, but the lack of formal safety guarantees still limits its application in safety-critical settings. A common practical approach is to augment the RL policy with a safety filter that overrides unsafe actions to prevent failures during both training and deployment. However, safety filtering is often perceived as sacrificing performance and hindering the learning process. We show that this perceived safety-performance tradeoff is not inherent and prove, for the first time, that enforcing safety with a sufficiently permissive safety filter does not degrade asymptotic performance. We formalize RL safety with a safety-critical Markov decision process (SC-MDP), which requires categorical, rather than high-probability, avoidance of catastrophic failure states. Additionally, we define an associated filtered MDP in which all actions result in safe effects, thanks to a safety filter that is considered to b

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyoh2025provably

Related papers

• Concurrent Learning Of Policy And Unknown Safety Constraints In Reinforcement Learning (2024)0.00
• Actsafe: Active Exploration With Safety Constraints For Reinforcement Learning (2024)0.00
• Safe Reinforcement Learning With Dual Robustness (2023)8.60
• On The Robustness Of Safe Reinforcement Learning Under Observational Perturbations (2022)0.00
• DEFENDER: Dtw-based Episode Filtering Using Demonstrations For Enhancing RL Safety (2023)0.00
• Safe Reinforcement Learning Via Projection On A Safe Set: How To Achieve Optimality? (2020)0.00
• Safety Modulation: Enhancing Safety In Reinforcement Learning Through Cost-modulated Rewards (2025)0.00
• Certifying Safety In Reinforcement Learning Under Adversarial Perturbation Attacks (2022)0.00