Autoeg: Automated Experience Grafting For Off-policy Deep Reinforcement Learning

Abstract

Deep reinforcement learning (RL) algorithms frequently require prohibitive interaction experience to ensure the quality of learned policies. The limitation is partly because the agent cannot learn much from the many low-quality trials in early learning phase, which results in low learning rate. Focusing on addressing this limitation, this paper makes a twofold contribution. First, we develop an algorithm, called Experience Grafting (EG), to enable RL agents to reorganize segments of the few high-quality trajectories from the experience pool to generate many synthetic trajectories while retaining the quality. Second, building on EG, we further develop an AutoEG agent that automatically learns to adjust the grafting-based learning strategy. Results collected from a set of six robotic control environments show that, in comparison to a standard deep RL algorithm (DDPG), AutoEG increases the speed of learning process by at least 30%.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keylu2020autoeg

Related papers

• Enhanced Experience Replay Generation For Efficient Reinforcement Learning (2017)0.00
• Experience Augmentation: Boosting And Accelerating Off-policy Multi-agent Reinforcement Learning (2020)0.00
• Asynchronous Episodic Deep Deterministic Policy Gradient: Towards Continuous Control In Computationally Complex Environments (2019)0.00
• Evolution-guided Policy Gradient In Reinforcement Learning (2018)0.00
• Control-optimized Deep Reinforcement Learning For Artificially Intelligent Autonomous Systems (2025)0.00
• Stabilising Experience Replay For Deep Multi-agent Reinforcement Learning (2017)0.00
• Adaptive Experience Selection For Policy Gradient (2020)0.00
• Sample-efficient Automated Deep Reinforcement Learning (2020)0.00