Joint Learning Of Policy With Unknown Temporal Constraints For Safe Reinforcement Learning

Abstract

In many real-world applications, safety constraints for reinforcement learning (RL) algorithms are either unknown or not explicitly defined. We propose a framework that concurrently learns safety constraints and optimal RL policies in such environments, supported by theoretical guarantees. Our approach merges a logically-constrained RL algorithm with an evolutionary algorithm to synthesize signal temporal logic (STL) specifications. The framework is underpinned by theorems that establish the convergence of our joint learning process and provide error bounds between the discovered policy and the true optimal policy. We showcased our framework in grid-world environments, successfully identifying both acceptable safety constraints and RL policies while demonstrating the effectiveness of our theorems in practice.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyyifru2023joint

Related papers

• Concurrent Learning Of Policy And Unknown Safety Constraints In Reinforcement Learning (2024)0.00
• Probabilistic Satisfaction Of Temporal Logic Constraints In Reinforcement Learning Via Adaptive Policy-switching (2024)0.00
• Model-based Safe Deep Reinforcement Learning Via A Constrained Proximal Policy Optimization Algorithm (2022)5.24
• Context-aware Safe Reinforcement Learning For Non-stationary Environments (2021)9.76
• Sample-efficient Reinforcement Learning With Temporal Logic Objectives: Leveraging The Task Specification To Guide Exploration (2024)0.00
• Learning Safe Policies With Expert Guidance (2018)0.00
• Policy Bifurcation In Safe Reinforcement Learning (2024)0.00
• Safe Reinforcement Learning With Dual Robustness (2023)8.60