Integrating Distributed Architectures In Highly Modular RL Libraries

Abstract

Advancing reinforcement learning (RL) requires tools that are flexible enough to easily prototype new methods while avoiding impractically slow experimental turnaround times. To match the first requirement, the most popular RL libraries advocate for highly modular agent composability, which facilitates experimentation and development. To solve challenging environments within reasonable time frames, scaling RL to large sampling and computing resources has proved a successful strategy. However, this capability has been so far difficult to combine with modularity. In this work, we explore design choices to allow agent composability both at a local and distributed level of execution. We propose a versatile approach that allows the definition of RL agents at different scales through independent reusable components. We demonstrate experimentally that our design choices allow us to reproduce classical benchmarks, explore multiple distributed architectures, and solve novel and complex environm

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keybou2020integrating

Related papers

• Rllib: Abstractions For Distributed Reinforcement Learning (2017)0.00
• Heterogeneous Knowledge For Augmented Modular Reinforcement Learning (2023)0.00
• SRL: Scaling Distributed Reinforcement Learning To Over Ten Thousand Cores (2023)0.00
• Acme: A Research Framework For Distributed Reinforcement Learning (2020)0.00
• The AI Arena: A Framework For Distributed Multi-agent Reinforcement Learning (2021)0.00
• Rllib Flow: Distributed Reinforcement Learning Is A Dataflow Problem (2020)0.00
• Distributed Multi-agent Reinforcement Learning Based On Graph-induced Local Value Functions (2022)4.52
• Influence-augmented Local Simulators: A Scalable Solution For Fast Deep RL In Large Networked Systems (2022)0.00