Highly Parallelized Reinforcement Learning Training With Relaxed Assignment Dependencies

Abstract

As the demands for superior agents grow, the training complexity of Deep Reinforcement Learning (DRL) becomes higher. Thus, accelerating training of DRL has become a major research focus. Dividing the DRL training process into subtasks and using parallel computation can effectively reduce training costs. However, current DRL training systems lack sufficient parallelization due to data assignment between subtask components. This assignment issue has been ignored, but addressing it can further boost training efficiency. Therefore, we propose a high-throughput distributed RL training system called TianJi. It relaxes assignment dependencies between subtask components and enables event-driven asynchronous communication. Meanwhile, TianJi maintains clear boundaries between subtask components. To address convergence uncertainty from relaxed assignment dependencies, TianJi proposes a distributed strategy based on the balance of sample production and consumption. The strategy controls the stale

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyhe2025highly

Related papers

• Decentralized Task Scheduling In Distributed Systems: A Deep Reinforcement Learning Approach (2026)0.00
• Distributed Deep Reinforcement Learning: An Overview (2020)0.00
• Acceleration For Deep Reinforcement Learning Using Parallel And Distributed Computing: A Survey (2024)8.82
• Accelerated Methods For Deep Reinforcement Learning (2018)0.00
• SRL: Scaling Distributed Reinforcement Learning To Over Ten Thousand Cores (2023)0.00
• Dynamic Sparse Training For Deep Reinforcement Learning (2021)0.00
• Rllib Flow: Distributed Reinforcement Learning Is A Dataflow Problem (2020)0.00
• Quantum-train-based Distributed Multi-agent Reinforcement Learning (2024)7.16