Is Optimal Transport Necessary For Inverse Reinforcement Learning?

Abstract

Inverse Reinforcement Learning (IRL) aims to recover a reward function from expert demonstrations. Recently, Optimal Transport (OT) methods have been successfully deployed to align trajectories and infer rewards. While OT-based methods have shown strong empirical results, they introduce algorithmic complexity, hyperparameter sensitivity, and require solving the OT optimization problems. In this work, we challenge the necessity of OT in IRL by proposing two simple, heuristic alternatives: (1) Minimum-Distance Reward, which assigns rewards based on the nearest expert state regardless of temporal order; and (2) Segment-Matching Reward, which incorporates lightweight temporal alignment by matching agent states to corresponding segments in the expert trajectory. These methods avoid optimization, exhibit linear-time complexity, and are easy to implement. Through extensive evaluations across 32 online and offline benchmarks with three reinforcement learning algorithms, we show that our simple

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keydong2025is

Related papers

• Understanding Reward Ambiguity Through Optimal Transport Theory In Inverse Reinforcement Learning (2023)0.00
• Can Optimal Transport Improve Federated Inverse Reinforcement Learning? (2026)0.00
• Imitation Learning From Observation Through Optimal Transport (2023)2.26
• Offline Inverse RL: New Solution Concepts And Provably Efficient Algorithms (2024)0.00
• Distance-rank Aware Sequential Reward Learning For Inverse Reinforcement Learning With Sub-optimal Demonstrations (2023)0.00
• Inverse Reinforcement Learning Without Reinforcement Learning (2023)0.00
• Towards Theoretical Understanding Of Inverse Reinforcement Learning (2023)0.00
• Is Inverse Reinforcement Learning Harder Than Standard Reinforcement Learning? A Theoretical Perspective (2023)0.00