Probabilistic Counterexample Guidance For Safer Reinforcement Learning (extended Version)

Abstract

Safe exploration aims at addressing the limitations of Reinforcement Learning (RL) in safety-critical scenarios, where failures during trial-and-error learning may incur high costs. Several methods exist to incorporate external knowledge or to use proximal sensor data to limit the exploration of unsafe states. However, reducing exploration risks in unknown environments, where an agent must discover safety threats during exploration, remains challenging. In this paper, we target the problem of safe exploration by guiding the training with counterexamples of the safety requirement. Our method abstracts both continuous and discrete state-space systems into compact abstract models representing the safety-relevant knowledge acquired by the agent during exploration. We then exploit probabilistic counterexample generation to construct minimal simulation submodels eliciting safety requirement violations, where the agent can efficiently train offline to refine its policy towards minimising the

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyji2023probabilistic

Related papers

• Actsafe: Active Exploration With Safety Constraints For Reinforcement Learning (2024)0.00
• Reinforcement Learning By Guided Safe Exploration (2023)5.24
• Provably Safe PAC-MDP Exploration Using Analogies (2020)0.00
• Learning To Explore When Mistakes Are Not Allowed (2025)0.00
• Implicit Safe Set Algorithm For Provably Safe Reinforcement Learning (2024)0.00
• Learning Safe Policies With Expert Guidance (2018)0.00
• Safe-support Q-learning: Learning Without Unsafe Exploration (2026)0.00
• Safe Reinforcement Learning In Black-box Environments Via Adaptive Shielding (2024)2.26