Provably Efficient Exploration In Inverse Constrained Reinforcement Learning

Abstract

Optimizing objective functions subject to constraints is fundamental in many real-world applications. However, these constraints are often not readily defined and must be inferred from expert agent behaviors, a problem known as Inverse Constraint Inference. Inverse Constrained Reinforcement Learning (ICRL) is a common solver for recovering feasible constraints in complex environments, relying on training samples collected from interactive environments. However, the efficacy and efficiency of current sampling strategies remain unclear. We propose a strategic exploration framework for sampling with guaranteed efficiency to bridge this gap. By defining the feasible cost set for ICRL problems, we analyze how estimation errors in transition dynamics and the expert policy influence the feasibility of inferred constraints. Based on this analysis, we introduce two exploratory algorithms to achieve efficient constraint inference via 1) dynamically reducing the bounded aggregate error of cost es

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyyue2024provably

Related papers

• Active Exploration For Inverse Reinforcement Learning (2022)0.00
• Controlling Underestimation Bias In Constrained Reinforcement Learning For Safe Exploration (2026)0.00
• Reinforcement Learning With Convex Constraints (2019)0.00
• Constraint Sampling Reinforcement Learning: Incorporating Expertise For Faster Learning (2021)0.00
• Offline Inverse RL: New Solution Concepts And Provably Efficient Algorithms (2024)0.00
• Towards Theoretical Understanding Of Inverse Reinforcement Learning (2023)0.00
• Conservative Exploration For Policy Optimization Via Off-policy Policy Evaluation (2023)0.00
• Provably Efficient Exploration In Constrained Reinforcement Learning:posterior Sampling Is All You Need (2023)0.00