Expert-free Online Transfer Learning In Multi-agent Reinforcement Learning

Abstract

Transfer learning in Reinforcement Learning (RL) has been widely studied to overcome training issues of Deep-RL, i.e., exploration cost, data availability and convergence time, by introducing a way to enhance training phase with external knowledge. Generally, knowledge is transferred from expert-agents to novices. While this fixes the issue for a novice agent, a good understanding of the task on expert agent is required for such transfer to be effective. As an alternative, in this paper we propose Expert-Free Online Transfer Learning (EF-OnTL), an algorithm that enables expert-free real-time dynamic transfer learning in multi-agent system. No dedicated expert exists, and transfer source agent and knowledge to be transferred are dynamically selected at each transfer step based on agents' performance and uncertainty. To improve uncertainty estimation, we also propose State Action Reward Next-State Random Network Distillation (sars-RND), an extension of RND that estimates uncertainty from

Stats

• citations2
• S2 citations—
• github stars0
• HF likes0
• heat score3.58
• arxiv keycastagna2023expert

Related papers

• Knowru: Knowledge Reusing Via Knowledge Distillation In Multi-agent Reinforcement Learning (2021)9.23
• An Efficient Transfer Learning Framework For Multiagent Reinforcement Learning (2020)0.00
• Improved Reinforcement Learning In Cooperative Multi-agent Environments Using Knowledge Transfer (2021)0.00
• On The Feasibility Of Cross-task Transfer With Model-based Reinforcement Learning (2022)0.00
• The Role Of Exploration For Task Transfer In Reinforcement Learning (2022)0.00
• Investigating The Role Of Model-based Learning In Exploration And Transfer (2023)0.00
• Contextual Policy Transfer In Reinforcement Learning Domains Via Deep Mixtures-of-experts (2020)0.00
• Adaptive Target Localization Under Uncertainty Using Multi-agent Deep Reinforcement Learning With Knowledge Transfer (2025)6.34