Enhancing Inverse Reinforcement Learning Through Encoding Dynamic Information In Reward Shaping

Abstract

In this paper, we aim to tackle the limitation of the Adversarial Inverse Reinforcement Learning (AIRL) method in stochastic environments where theoretical results cannot hold and performance is degraded. To address this issue, we propose a novel method which infuses the dynamics information into the reward shaping with the theoretical guarantee for the induced optimal policy in the stochastic environments. Incorporating our novel model-enhanced rewards, we present a novel Model-Enhanced AIRL framework, which integrates transition model estimation directly into reward shaping. Furthermore, we provide a comprehensive theoretical analysis of the reward error bound and performance difference bound for our method. The experimental results in MuJoCo benchmarks show that our method can achieve superior performance in stochastic environments and competitive performance in deterministic environments, with significant improvement in sample efficiency, compared to existing baselines.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhan2024enhancing

Related papers

• Learning Robust Rewards With Adversarial Inverse Reinforcement Learning (2017)0.00
• Automatic Intrinsic Reward Shaping For Exploration In Deep Reinforcement Learning (2023)0.00
• Highly Efficient Self-adaptive Reward Shaping For Reinforcement Learning (2024)0.00
• Multimodal Reward Shaping For Efficient Exploration In Reinforcement Learning (2021)0.00
• BAMDP Shaping: A Unified Framework For Intrinsic Motivation And Reward Shaping (2024)0.00
• Shaping Sparse Rewards In Reinforcement Learning: A Semi-supervised Approach (2025)0.00
• Influencing Reinforcement Learning Through Natural Language Guidance (2021)0.00
• Efficient Inference For Inverse Reinforcement Learning And Dynamic Discrete Choice Models (2025)0.00