CAPSULE: Control-theoretic Action Perturbations For Safe Uncertainty-aware Reinforcement Learning

Abstract

arXiv:2604.23576v1 Announce Type: cross Abstract: Ensuring safe exploration in high-dimensional systems with unknown dynamics remains a significant challenge. Existing safe reinforcement learning methods often provide safety guarantees only in expectation, which can still lead to safety violations. Control-theoretic approaches, in contrast, offer hard constraint-based safety guarantees but typically assume access to known system dynamics or require accurate estimation of control-affine models. In this paper, we propose a safe reinforcement learning framework that learns a probabilistic control-affine dynamics model in an offline setting. The learned model is leveraged to explicitly construct control barrier functions (CBFs) that incorporate model uncertainty to provide conservative safety constraints. These CBF constraints are enforced through an online constraint-based action correction mechanism, enabling safe exploration without overly restricting task performance. Empirical evalua

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keynarava2026capsule

Related papers

• Conservative And Adaptive Penalty For Model-based Safe Reinforcement Learning (2021)0.00
• Dyna-style Safety Augmented Reinforcement Learning: Staying Safe In The Face Of Uncertainty (2026)0.00
• Actsafe: Active Exploration With Safety Constraints For Reinforcement Learning (2024)0.00
• Optimal Transport Perturbations For Safe Reinforcement Learning With Robustness Guarantees (2023)0.00
• DOPE: Doubly Optimistic And Pessimistic Exploration For Safe Reinforcement Learning (2021)0.00
• Safe-support Q-learning: Learning Without Unsafe Exploration (2026)0.00
• Safe Exploration Using Bayesian World Models And Log-barrier Optimization (2024)0.00
• Context-aware Safe Reinforcement Learning For Non-stationary Environments (2021)9.76