Model Selection For Inverse Reinforcement Learning Via Structural Risk Minimization

Abstract

Inverse reinforcement learning (IRL) usually assumes the reward function model is pre-specified as a weighted sum of features and estimates the weighting parameters only. However, how to select features and determine a proper reward model is nontrivial and experience-dependent. A simplistic model is less likely to contain the ideal reward function, while a model with high complexity leads to substantial computation cost and potential overfitting. This paper addresses this trade-off in the model selection for IRL problems by introducing the structural risk minimization (SRM) framework from statistical learning. SRM selects an optimal reward function class from a hypothesis set minimizing both estimation error and model complexity. To formulate an SRM scheme for IRL, we estimate the policy gradient from given demonstration as the empirical risk, and establish the upper bound of Rademacher complexity as the model penalty of hypothesis function classes. The SRM learning guarantee is furthe

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyqu2023model

Related papers

• Misspecification In Inverse Reinforcement Learning (2022)5.24
• Modeling And Interpreting Real-world Human Risk Decision Making With Inverse Reinforcement Learning (2019)0.00
• Towards Theoretical Understanding Of Inverse Reinforcement Learning (2023)0.00
• Partial Identifiability And Misspecification In Inverse Reinforcement Learning (2024)0.00
• Quantifying The Sensitivity Of Inverse Reinforcement Learning To Misspecification (2024)0.00
• Inverse Reinforcement Learning With Simultaneous Estimation Of Rewards And Dynamics (2016)0.00
• Active Learning For Risk-sensitive Inverse Reinforcement Learning (2019)0.00
• Maximum-likelihood Inverse Reinforcement Learning With Finite-time Guarantees (2022)0.00