Actsafe: Active Exploration With Safety Constraints For Reinforcement Learning

Abstract

Reinforcement learning (RL) is ubiquitous in the development of modern AI systems. However, state-of-the-art RL agents require extensive, and potentially unsafe, interactions with their environments to learn effectively. These limitations confine RL agents to simulated environments, hindering their ability to learn directly in real-world settings. In this work, we present ActSafe, a novel model-based RL algorithm for safe and efficient exploration. ActSafe learns a well-calibrated probabilistic model of the system and plans optimistically w.r.t. the epistemic uncertainty about the unknown dynamics, while enforcing pessimism w.r.t. the safety constraints. Under regularity assumptions on the constraints and dynamics, we show that ActSafe guarantees safety during learning while also obtaining a near-optimal policy in finite time. In addition, we propose a practical variant of ActSafe that builds on latest model-based RL advancements and enables safe exploration even in high-dimensional se

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyas2024actsafe

Related papers

• Implicit Safe Set Algorithm For Provably Safe Reinforcement Learning (2024)0.00
• Safe-support Q-learning: Learning Without Unsafe Exploration (2026)0.00
• Reinforcement Learning By Guided Safe Exploration (2023)5.24
• Omnisafe: An Infrastructure For Accelerating Safe Reinforcement Learning Research (2023)0.00
• Probabilistic Counterexample Guidance For Safer Reinforcement Learning (extended Version) (2023)0.00
• Safe Reinforcement Learning In Black-box Environments Via Adaptive Shielding (2024)2.26
• Safe Continual Reinforcement Learning In Non-stationary Environments (2026)12.89
• Context-aware Safe Reinforcement Learning For Non-stationary Environments (2021)9.76