Implicit Safe Set Algorithm For Provably Safe Reinforcement Learning

Abstract

Deep reinforcement learning (DRL) has demonstrated remarkable performance in many continuous control tasks. However, a significant obstacle to the real-world application of DRL is the lack of safety guarantees. Although DRL agents can satisfy system safety in expectation through reward shaping, designing agents to consistently meet hard constraints (e.g., safety specifications) at every time step remains a formidable challenge. In contrast, existing work in the field of safe control provides guarantees on persistent satisfaction of hard safety constraints. However, these methods require explicit analytical system dynamics models to synthesize safe control, which are typically inaccessible in DRL settings. In this paper, we present a model-free safe control algorithm, the implicit safe set algorithm, for synthesizing safeguards for DRL agents that ensure provable safety throughout training. The proposed algorithm synthesizes a safety index (barrier certificate) and a subsequent safe con

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhao2024implicit

Related papers

• Actsafe: Active Exploration With Safety Constraints For Reinforcement Learning (2024)0.00
• Safe Continual Reinforcement Learning In Non-stationary Environments (2026)12.89
• Safe Reinforcement Learning Via Projection On A Safe Set: How To Achieve Optimality? (2020)0.00
• Provably Optimal Reinforcement Learning Under Safety Filtering (2025)0.00
• Towards Safe Reinforcement Learning Via Constraining Conditional Value-at-risk (2022)0.00
• DOPE: Doubly Optimistic And Pessimistic Exploration For Safe Reinforcement Learning (2021)0.00
• Probabilistic Counterexample Guidance For Safer Reinforcement Learning (extended Version) (2023)0.00
• Model-based Safe Deep Reinforcement Learning Via A Constrained Proximal Policy Optimization Algorithm (2022)5.24