Dyna-style Safety Augmented Reinforcement Learning: Staying Safe In The Face Of Uncertainty

Abstract

arXiv:2604.25508v1 Announce Type: new Abstract: Safety remains an open problem in reinforcement learning (RL), especially during training. While safety filters are promising to address safe exploration, they are generally poorly suited for high-dimensional systems with unknown dynamics. We propose Dyna-style Safety Augmented Reinforcement Learning (Dyna-SAuR), a novel algorithm that learns both a scalable safety filter and a control policy using a learned uncertainty-aware dynamics model, while requiring minimal domain knowledge. The filter avoids failures and high uncertainty regions. Thus, better models expand the set of safe and certain states, reducing filter conservatism. We present the effectiveness of Dyna-SAuR on goal-reaching CartPole as well as MuJoCo Walker, reducing failures compared to state-of-the-art methods by 2 orders of magnitude.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyeisele2026dyna

Related papers

• Provably Optimal Reinforcement Learning Under Safety Filtering (2025)0.00
• Actsafe: Active Exploration With Safety Constraints For Reinforcement Learning (2024)0.00
• CAPSULE: Control-theoretic Action Perturbations For Safe Uncertainty-aware Reinforcement Learning (2026)0.00
• Implicit Safe Set Algorithm For Provably Safe Reinforcement Learning (2024)0.00
• Reinforcement Learning With Adaptive Regularization For Safe Control Of Critical Systems (2024)0.00
• Safety Correction From Baseline: Towards The Risk-aware Policy In Robotics Via Dual-agent Reinforcement Learning (2022)3.58
• DOPE: Doubly Optimistic And Pessimistic Exploration For Safe Reinforcement Learning (2021)0.00
• Safe-support Q-learning: Learning Without Unsafe Exploration (2026)0.00