Selective Reincarnation: Offline-to-online Multi-agent Reinforcement Learning

Abstract

'Reincarnation' in reinforcement learning has been proposed as a formalisation of reusing prior computation from past experiments when training an agent in an environment. In this paper, we present a brief foray into the paradigm of reincarnation in the multi-agent (MA) context. We consider the case where only some agents are reincarnated, whereas the others are trained from scratch -- selective reincarnation. In the fully-cooperative MA setting with heterogeneous agents, we demonstrate that selective reincarnation can lead to higher returns than training fully from scratch, and faster convergence than training with full reincarnation. However, the choice of which agents to reincarnate in a heterogeneous system is vitally important to the outcome of the training -- in fact, a poor choice can lead to considerably worse results than the alternatives. We argue that a rich field of work exists here, and we hope that our effort catalyses further energy in bringing the topic of reincarnation

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyformanek2023selective

Related papers

• Reincarnating Reinforcement Learning: Reusing Prior Computation To Accelerate Progress (2022)0.00
• Offline-to-online Multi-agent Reinforcement Learning With Offline Value Function Memory And Sequential Exploration (2024)2.26
• Plan Better Amid Conservatism: Offline Multi-agent Reinforcement Learning With Actor Rectification (2021)0.00
• Learning From Good Trajectories In Offline Multi-agent Reinforcement Learning (2022)5.24
• Offline Decentralized Multi-agent Reinforcement Learning (2021)7.50
• Deep Multiagent Reinforcement Learning: Challenges And Directions (2021)0.00
• Improved Training Mechanism For Reinforcement Learning Via Online Model Selection (2025)0.00
• Believe What You See: Implicit Constraint Approach For Offline Multi-agent Reinforcement Learning (2021)0.00