Learning Behavioral Soft Constraints From Demonstrations

Abstract

Many real-life scenarios require humans to make difficult trade-offs: do we always follow all the traffic rules or do we violate the speed limit in an emergency? These scenarios force us to evaluate the trade-off between collective rules and norms with our own personal objectives and desires. To create effective AI-human teams, we must equip AI agents with a model of how humans make these trade-offs in complex environments when there are implicit and explicit rules and constraints. Agent equipped with these models will be able to mirror human behavior and/or to draw human attention to situations where decision making could be improved. To this end, we propose a novel inverse reinforcement learning (IRL) method: Max Entropy Inverse Soft Constraint IRL (MESC-IRL), for learning implicit hard and soft constraints over states, actions, and state features from demonstrations in deterministic and non-deterministic environments modeled as Markov Decision Processes (MDPs). Our method enables ag

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyglazier2022learning

Related papers

• Multi-agent Inverse Reinforcement Learning: Suboptimal Demonstrations And Alternative Solution Concepts (2021)0.00
• Accounting For Human Learning When Inferring Human Preferences (2020)0.00
• Active Learning For Risk-sensitive Inverse Reinforcement Learning (2019)0.00
• Task-guided Inverse Reinforcement Learning Under Partial Information (2021)0.00
• Inverse Reinforcement Learning With Simultaneous Estimation Of Rewards And Dynamics (2016)0.00
• Provably Efficient Exploration In Inverse Constrained Reinforcement Learning (2024)0.00
• Inverse Reinforcement Learning With Explicit Policy Estimates (2021)2.26
• When Demonstrations Meet Generative World Models: A Maximum Likelihood Framework For Offline Inverse Reinforcement Learning (2023)0.00