Reinforcement Learning By Guided Safe Exploration

Abstract

Safety is critical to broadening the application of reinforcement learning (RL). Often, we train RL agents in a controlled environment, such as a laboratory, before deploying them in the real world. However, the real-world target task might be unknown prior to deployment. Reward-free RL trains an agent without the reward to adapt quickly once the reward is revealed. We consider the constrained reward-free setting, where an agent (the guide) learns to explore safely without the reward signal. This agent is trained in a controlled environment, which allows unsafe interactions and still provides the safety signal. After the target task is revealed, safety violations are not allowed anymore. Thus, the guide is leveraged to compose a safe behaviour policy. Drawing from transfer learning, we also regularize a target policy (the student) towards the guide while the student is unreliable and gradually eliminate the influence of the guide as training progresses. The empirical analysis shows tha

Stats

• citations4
• S2 citations—
• github stars0
• HF likes0
• heat score5.24
• arxiv keyyang2023reinforcement

Related papers

• Learning To Explore When Mistakes Are Not Allowed (2025)0.00
• Actsafe: Active Exploration With Safety Constraints For Reinforcement Learning (2024)0.00
• Probabilistic Counterexample Guidance For Safer Reinforcement Learning (extended Version) (2023)0.00
• Safe-support Q-learning: Learning Without Unsafe Exploration (2026)0.00
• Learning Safe Policies With Expert Guidance (2018)0.00
• Safe Reinforcement Learning In Black-box Environments Via Adaptive Shielding (2024)2.26
• Guided Online Distillation: Promoting Safe Reinforcement Learning By Offline Demonstration (2023)4.52
• Safe Continual Reinforcement Learning In Non-stationary Environments (2026)12.89