Is Inverse Reinforcement Learning Harder Than Standard Reinforcement Learning? A Theoretical Perspective

Abstract

Inverse Reinforcement Learning (IRL) -- the problem of learning reward functions from demonstrations of an *expert policy* -- plays a critical role in developing intelligent systems. While widely used in applications, theoretical understandings of IRL present unique challenges and remain less developed compared with standard RL. For example, it remains open how to do IRL efficiently in standard *offline* settings with pre-collected data, where states are obtained from a *behavior policy* (which could be the expert policy itself), and actions are sampled from the expert policy. This paper provides the first line of results for efficient IRL in vanilla offline and online settings using polynomial samples and runtime. Our algorithms and analyses seamlessly adapt the pessimism principle commonly used in offline RL, and achieve IRL guarantees in stronger metrics than considered in existing work. We provide lower bounds showing that our sample complexities are nearly optimal. As an applica

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhao2023is

Related papers

• Inverse Reinforcement Learning Without Reinforcement Learning (2023)0.00
• Offline Inverse RL: New Solution Concepts And Provably Efficient Algorithms (2024)0.00
• Towards Theoretical Understanding Of Inverse Reinforcement Learning (2023)0.00
• A Survey Of Inverse Reinforcement Learning: Challenges, Methods And Progress (2018)0.00
• On The Effective Horizon Of Inverse Reinforcement Learning (2023)0.00
• Active Exploration For Inverse Reinforcement Learning (2022)0.00
• When Demonstrations Meet Generative World Models: A Maximum Likelihood Framework For Offline Inverse Reinforcement Learning (2023)0.00
• Task-guided Inverse Reinforcement Learning Under Partial Information (2021)0.00