MULTIPOLAR: Multi-source Policy Aggregation For Transfer Reinforcement Learning Between Diverse Environmental Dynamics

Abstract

Transfer reinforcement learning (RL) aims at improving the learning efficiency of an agent by exploiting knowledge from other source agents trained on relevant tasks. However, it remains challenging to transfer knowledge between different environmental dynamics without having access to the source environments. In this work, we explore a new challenge in transfer RL, where only a set of source policies collected under diverse unknown dynamics is available for learning a target task efficiently. To address this problem, the proposed approach, MULTI-source POLicy AggRegation (MULTIPOLAR), comprises two key techniques. We learn to aggregate the actions provided by the source policies adaptively to maximize the target task performance. Meanwhile, we learn an auxiliary network that predicts residuals around the aggregated actions, which ensures the target policy's expressiveness even when some of the source policies perform poorly. We demonstrated the effectiveness of MULTIPOLAR through an e

Stats

• citations10
• S2 citations—
• github stars0
• HF likes0
• heat score7.81
• arxiv keybarekatain2019multipolar

Related papers

• IOB: Integrating Optimization Transfer And Behavior Transfer For Multi-policy Reuse (2023)5.24
• Single Episode Policy Transfer In Reinforcement Learning (2019)0.00
• An Advantage Based Policy Transfer Algorithm For Reinforcement Learning With Measures Of Transferability (2023)0.00
• Diversity For Contingency: Learning Diverse Behaviors For Efficient Adaptation And Transfer (2023)0.00
• Contextual Policy Transfer In Reinforcement Learning Domains Via Deep Mixtures-of-experts (2020)0.00
• A Decentralized Policy Gradient Approach To Multi-task Reinforcement Learning (2020)0.00
• An Efficient Transfer Learning Framework For Multiagent Reinforcement Learning (2020)0.00
• A Policy Gradient Algorithm For Learning To Learn In Multiagent Reinforcement Learning (2020)0.00