A Dual Approach To Imitation Learning From Observations With Offline Datasets

Abstract

Demonstrations are an effective alternative to task specification for learning agents in settings where designing a reward function is difficult. However, demonstrating expert behavior in the action space of the agent becomes unwieldy when robots have complex, unintuitive morphologies. We consider the practical setting where an agent has a dataset of prior interactions with the environment and is provided with observation-only expert demonstrations. Typical learning from observations approaches have required either learning an inverse dynamics model or a discriminator as intermediate steps of training. Errors in these intermediate one-step models compound during downstream policy learning or deployment. We overcome these limitations by directly learning a multi-step utility function that quantifies how each action impacts the agent's divergence from the expert's visitation distribution. Using the principle of duality, we derive DILO (Dual Imitation Learning from Observations), an algor

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keysikchi2024a

Related papers

• A Simple Solution For Offline Imitation From Observations And Examples With Possibly Incomplete Trajectories (2023)0.00
• Offline Imitation Learning With Suboptimal Demonstrations Via Relaxed Distribution Matching (2023)6.77
• When Demonstrations Meet Generative World Models: A Maximum Likelihood Framework For Offline Inverse Reinforcement Learning (2023)0.00
• Imitation Learning From Observation With Automatic Discount Scheduling (2023)0.00
• Co-imitation Learning Without Expert Demonstration (2021)0.00
• DITTO: Offline Imitation Learning With World Models (2023)0.00
• Lobsdice: Offline Learning From Observation Via Stationary Distribution Correction Estimation (2022)0.00
• Beyond-expert Performance With Limited Demonstrations: Efficient Imitation Learning With Double Exploration (2025)0.00