Imitation Learning Via Differentiable Physics

Abstract

Existing imitation learning (IL) methods such as inverse reinforcement learning (IRL) usually have a double-loop training process, alternating between learning a reward function and a policy and tend to suffer long training time and high variance. In this work, we identify the benefits of differentiable physics simulators and propose a new IL method, i.e., Imitation Learning via Differentiable Physics (ILD), which gets rid of the double-loop design and achieves significant improvements in final performance, convergence speed, and stability. The proposed ILD incorporates the differentiable physics simulator as a physics prior into its computational graph for policy learning. It unrolls the dynamics by sampling actions from a parameterized policy, simply minimizing the distance between the expert trajectory and the agent trajectory, and back-propagating the gradient into the policy via temporal physics operators. With the physics prior, ILD policies can not only be transferable to unseen

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keychen2022imitation

Related papers

• Iq-learn: Inverse Soft-q Learning For Imitation (2021)0.00
• RLIF: Interactive Imitation Learning As Reinforcement Learning (2023)0.00
• State-only Imitation With Transition Dynamics Mismatch (2020)0.00
• Inverse Reinforcement Learning With Simultaneous Estimation Of Rewards And Dynamics (2016)0.00
• Explaining Fast Improvement In Online Imitation Learning (2020)0.00
• Imitation Learning From Observation Through Optimal Transport (2023)2.26
• FP-IRL: Fokker-planck Inverse Reinforcement Learning -- A Physics-constrained Approach To Markov Decision Processes (2023)2.26
• Imitation Learning In Discounted Linear Mdps Without Exploration Assumptions (2024)0.00