Towards Theoretical Understanding Of Inverse Reinforcement Learning

Abstract

Inverse reinforcement learning (IRL) denotes a powerful family of algorithms for recovering a reward function justifying the behavior demonstrated by an expert agent. A well-known limitation of IRL is the ambiguity in the choice of the reward function, due to the existence of multiple rewards that explain the observed behavior. This limitation has been recently circumvented by formulating IRL as the problem of estimating the feasible reward set, i.e., the region of the rewards compatible with the expert's behavior. In this paper, we make a step towards closing the theory gap of IRL in the case of finite-horizon problems with a generative model. We start by formally introducing the problem of estimating the feasible reward set, the corresponding PAC requirement, and discussing the properties of particular classes of rewards. Then, we provide the first minimax lower bound on the sample complexity for the problem of estimating the feasible reward set of order \(\{Ω\}\Bigl( \frac\{H^3SA\}\

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keymetelli2023towards

Related papers

• Offline Inverse RL: New Solution Concepts And Provably Efficient Algorithms (2024)0.00
• Maximum-likelihood Inverse Reinforcement Learning With Finite-time Guarantees (2022)0.00
• A Survey Of Inverse Reinforcement Learning: Challenges, Methods And Progress (2018)0.00
• Is Inverse Reinforcement Learning Harder Than Standard Reinforcement Learning? A Theoretical Perspective (2023)0.00
• Inverse Reinforcement Learning Without Reinforcement Learning (2023)0.00
• Partial Identifiability And Misspecification In Inverse Reinforcement Learning (2024)0.00
• On The Effective Horizon Of Inverse Reinforcement Learning (2023)0.00
• Inverse Reinforcement Learning With Simultaneous Estimation Of Rewards And Dynamics (2016)0.00