Reincarnating Reinforcement Learning: Reusing Prior Computation To Accelerate Progress

Abstract

Learning tabula rasa, that is without any prior knowledge, is the prevalent workflow in reinforcement learning (RL) research. However, RL systems, when applied to large-scale settings, rarely operate tabula rasa. Such large-scale systems undergo multiple design or algorithmic changes during their development cycle and use ad hoc approaches for incorporating these changes without re-training from scratch, which would have been prohibitively expensive. Additionally, the inefficiency of deep RL typically excludes researchers without access to industrial-scale resources from tackling computationally-demanding problems. To address these issues, we present reincarnating RL as an alternative workflow or class of problem settings, where prior computational work (e.g., learned policies) is reused or transferred between design iterations of an RL agent, or from one RL agent to another. As a step towards enabling reincarnating RL from any agent to any other agent, we focus on the specific setting

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyagarwal2022reincarnating

Related papers

• Selective Reincarnation: Offline-to-online Multi-agent Reinforcement Learning (2023)0.00
• Human-inspired Framework To Accelerate Reinforcement Learning (2023)0.00
• Snapshot Reinforcement Learning: Leveraging Prior Trajectories For Efficiency (2024)0.00
• AWAC: Accelerating Online Reinforcement Learning With Offline Datasets (2020)0.00
• Interactive Reinforcement Learning With Dynamic Reuse Of Prior Knowledge From Human/agent's Demonstration (2018)8.60
• Knowru: Knowledge Reusing Via Knowledge Distillation In Multi-agent Reinforcement Learning (2021)9.23
• Replacing Rewards With Examples: Example-based Policy Search Via Recursive Classification (2021)0.00
• Learning To Reinforcement Learn (2016)0.00