Off-dynamics Reinforcement Learning: Training For Transfer With Domain Classifiers

Abstract

We propose a simple, practical, and intuitive approach for domain adaptation in reinforcement learning. Our approach stems from the idea that the agent's experience in the source domain should look similar to its experience in the target domain. Building off of a probabilistic view of RL, we formally show that we can achieve this goal by compensating for the difference in dynamics by modifying the reward function. This modified reward function is simple to estimate by learning auxiliary classifiers that distinguish source-domain transitions from target-domain transitions. Intuitively, the modified reward function penalizes the agent for visiting states and taking actions in the source domain which are not possible in the target domain. Said another way, the agent is penalized for transitions that would indicate that the agent is interacting with the source domain, rather than the target domain. Our approach is applicable to domains with continuous states and actions and does not requir

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyeysenbach2020off

Related papers

• Unsupervised Domain Adaptation With Dynamics-aware Rewards In Reinforcement Learning (2021)0.00
• Off-dynamics Reinforcement Learning Via Domain Adaptation And Reward Augmented Imitation (2024)0.00
• Adarl: What, Where, And How To Adapt In Transfer Reinforcement Learning (2021)0.00
• A Conservative Approach For Few-shot Transfer In Off-dynamics Reinforcement Learning (2023)0.00
• DARLA: Improving Zero-shot Transfer In Reinforcement Learning (2017)0.00
• Transferable Reward Learning By Dynamics-agnostic Discriminator Ensemble (2022)0.00
• Self-supervised Reinforcement Learning That Transfers Using Random Features (2023)2.26
• Efficient Cross-domain Offline Reinforcement Learning With Dynamics- And Value-aligned Data Filtering (2025)0.00