Accelerated Methods For Deep Reinforcement Learning

Abstract

Deep reinforcement learning (RL) has achieved many recent successes, yet experiment turn-around time remains a key bottleneck in research and in practice. We investigate how to optimize existing deep RL algorithms for modern computers, specifically for a combination of CPUs and GPUs. We confirm that both policy gradient and Q-value learning algorithms can be adapted to learn using many parallel simulator instances. We further find it possible to train using batch sizes considerably larger than are standard, without negatively affecting sample complexity or final performance. We leverage these facts to build a unified framework for parallelization that dramatically hastens experiments in both classes of algorithm. All neural network computations use GPUs, accelerating both data collection and training. Our results include using an entire DGX-1 to learn successful strategies in Atari games in mere minutes, using both synchronous and asynchronous algorithms.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keystooke2018accelerated

Related papers

• Acceleration For Deep Reinforcement Learning Using Parallel And Distributed Computing: A Survey (2024)8.82
• Deep Q-networks For Accelerating The Training Of Deep Neural Networks (2016)0.00
• Parallel \(q\)-learning: Scaling Off-policy Reinforcement Learning Under Massively Parallel Simulation (2023)0.00
• Importance Of Using Appropriate Baselines For Evaluation Of Data-efficiency In Deep Reinforcement Learning For Atari (2020)0.00
• Efficient Off-policy Reinforcement Learning Via Brain-inspired Computing (2022)8.35
• Human-inspired Framework To Accelerate Reinforcement Learning (2023)0.00
• Influence-augmented Local Simulators: A Scalable Solution For Fast Deep RL In Large Networked Systems (2022)0.00
• Hyperparameter Tuning For Deep Reinforcement Learning Applications (2022)0.00