Influence-augmented Local Simulators: A Scalable Solution For Fast Deep RL In Large Networked Systems

Abstract

Learning effective policies for real-world problems is still an open challenge for the field of reinforcement learning (RL). The main limitation being the amount of data needed and the pace at which that data can be obtained. In this paper, we study how to build lightweight simulators of complicated systems that can run sufficiently fast for deep RL to be applicable. We focus on domains where agents interact with a reduced portion of a larger environment while still being affected by the global dynamics. Our method combines the use of local simulators with learned models that mimic the influence of the global system. The experiments reveal that incorporating this idea into the deep RL workflow can considerably accelerate the training process and presents several opportunities for the future.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keysuau2022influence

Related papers

• Distributed Influence-augmented Local Simulators For Parallel MARL In Large Networked Systems (2022)0.00
• Overcoming The Sim-to-real Gap: Leveraging Simulation To Learn To Explore For Real-world RL (2024)5.84
• Scalable Semantic Non-markovian Simulation Proxy For Reinforcement Learning (2023)0.00
• Accelerated Methods For Deep Reinforcement Learning (2018)0.00
• When To Trust Your Simulator: Dynamics-aware Hybrid Offline-and-online Reinforcement Learning (2022)2.26
• Persim: Data-efficient Offline Reinforcement Learning With Heterogeneous Agents Via Personalized Simulators (2021)0.00
• Policy Learning For Off-dynamics RL With Deficient Support (2024)0.00
• Scalable Reinforcement Learning For Multi-agent Networked Systems (2019)10.35