Task-guided Inverse Reinforcement Learning Under Partial Information

Abstract

We study the problem of inverse reinforcement learning (IRL), where the learning agent recovers a reward function using expert demonstrations. Most of the existing IRL techniques make the often unrealistic assumption that the agent has access to full information about the environment. We remove this assumption by developing an algorithm for IRL in partially observable Markov decision processes (POMDPs). The algorithm addresses several limitations of existing techniques that do not take the information asymmetry between the expert and the learner into account. First, it adopts causal entropy as the measure of the likelihood of the expert demonstrations as opposed to entropy in most existing IRL techniques, and avoids a common source of algorithmic complexity. Second, it incorporates task specifications expressed in temporal logic into IRL. Such specifications may be interpreted as side information available to the learner a priori in addition to the demonstrations and may reduce the inf

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keydjeumou2021task

Related papers

• Task-guided IRL In Pomdps That Scales (2022)2.26
• Inverse Reinforcement Learning Without Reinforcement Learning (2023)0.00
• Inverse Reinforcement Learning With Missing Data (2019)0.00
• Towards Theoretical Understanding Of Inverse Reinforcement Learning (2023)0.00
• Is Inverse Reinforcement Learning Harder Than Standard Reinforcement Learning? A Theoretical Perspective (2023)0.00
• Reinforcement Learning Under Partial Observability Guided By Learned Environment Models (2022)6.34
• Partial Identifiability And Misspecification In Inverse Reinforcement Learning (2024)0.00
• Active Exploration For Inverse Reinforcement Learning (2022)0.00