A Bayesian Approach To Robust Inverse Reinforcement Learning

Abstract

We consider a Bayesian approach to offline model-based inverse reinforcement learning (IRL). The proposed framework differs from existing offline model-based IRL approaches by performing simultaneous estimation of the expert's reward function and subjective model of environment dynamics. We make use of a class of prior distributions which parameterizes how accurate the expert's model of the environment is to develop efficient algorithms to estimate the expert's reward and subjective dynamics in high-dimensional settings. Our analysis reveals a novel insight that the estimated policy exhibits robust performance when the expert is believed (a priori) to have a highly accurate model of the environment. We verify this observation in the MuJoCo environments and show that our algorithms outperform state-of-the-art offline IRL algorithms.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keywei2023a

Related papers

• When Demonstrations Meet Generative World Models: A Maximum Likelihood Framework For Offline Inverse Reinforcement Learning (2023)0.00
• Offline Inverse RL: New Solution Concepts And Provably Efficient Algorithms (2024)0.00
• Distributional Inverse Reinforcement Learning (2025)0.00
• Bayesian Robust Optimization For Imitation Learning (2020)0.00
• Kernel Density Bayesian Inverse Reinforcement Learning (2023)0.00
• CLARE: Conservative Model-based Reward Learning For Offline Inverse Reinforcement Learning (2023)0.00
• Towards Theoretical Understanding Of Inverse Reinforcement Learning (2023)0.00
• Active Exploration For Inverse Reinforcement Learning (2022)0.00