Offline Inverse RL: New Solution Concepts And Provably Efficient Algorithms

Abstract

Inverse reinforcement learning (IRL) aims to recover the reward function of an expert agent from demonstrations of behavior. It is well-known that the IRL problem is fundamentally ill-posed, i.e., many reward functions can explain the demonstrations. For this reason, IRL has been recently reframed in terms of estimating the feasible reward set (Metelli et al., 2021), thus, postponing the selection of a single reward. However, so far, the available formulations and algorithmic solutions have been proposed and analyzed mainly for the online setting, where the learner can interact with the environment and query the expert at will. This is clearly unrealistic in most practical applications, where the availability of an offline dataset is a much more common scenario. In this paper, we introduce a novel notion of feasible reward set capturing the opportunities and limitations of the offline setting and we analyze the complexity of its estimation. This requires the introduction an original le

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keylazzati2024offline

Related papers

• Is Inverse Reinforcement Learning Harder Than Standard Reinforcement Learning? A Theoretical Perspective (2023)0.00
• Towards Theoretical Understanding Of Inverse Reinforcement Learning (2023)0.00
• When Demonstrations Meet Generative World Models: A Maximum Likelihood Framework For Offline Inverse Reinforcement Learning (2023)0.00
• Inverse Reinforcement Learning Without Reinforcement Learning (2023)0.00
• A Survey Of Inverse Reinforcement Learning: Challenges, Methods And Progress (2018)0.00
• CLARE: Conservative Model-based Reward Learning For Offline Inverse Reinforcement Learning (2023)0.00
• A Bayesian Approach To Robust Inverse Reinforcement Learning (2023)0.00
• Distributional Inverse Reinforcement Learning (2025)0.00