Contextual Policy Transfer In Reinforcement Learning Domains Via Deep Mixtures-of-experts

Abstract

In reinforcement learning, agents that consider the context, or current state, when selecting source policies for transfer have been shown to outperform context-free approaches. However, none of the existing approaches transfer knowledge contextually from model-based learners to a model-free learner. This could be useful, for instance, when source policies are intentionally learned on diverse simulations with plentiful data but transferred to a real-world setting with limited data. In this paper, we assume knowledge of estimated source task dynamics and policies, and common sub-goals but different dynamics. We introduce a novel deep mixture-of-experts formulation for learning state-dependent beliefs over source task dynamics that match the target dynamics using state trajectories collected from the target task. The mixture model is easy to interpret, demonstrates robustness to estimation errors in dynamics, and is compatible with most learning algorithms. We then show how this model ca

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keygimelfarb2020contextual

Related papers

• MULTIPOLAR: Multi-source Policy Aggregation For Transfer Reinforcement Learning Between Diverse Environmental Dynamics (2019)7.81
• Investigating The Role Of Model-based Learning In Exploration And Transfer (2023)0.00
• Probabilistic Mixture-of-experts For Efficient Deep Reinforcement Learning (2021)0.00
• Similarity-based Knowledge Transfer For Cross-domain Reinforcement Learning (2023)0.00
• Continuous Action Reinforcement Learning From A Mixture Of Interpretable Experts (2020)0.00
• IOB: Integrating Optimization Transfer And Behavior Transfer For Multi-policy Reuse (2023)5.24
• Learning Without Knowing: Unobserved Context In Continuous Transfer Reinforcement Learning (2021)0.00
• Transfer Reinforcement Learning In Heterogeneous Action Spaces Using Subgoal Mapping (2024)0.00