Delay-aware Multi-agent Reinforcement Learning For Cooperative And Competitive Environments

Abstract

Action and observation delays exist prevalently in the real-world cyber-physical systems which may pose challenges in reinforcement learning design. It is particularly an arduous task when handling multi-agent systems where the delay of one agent could spread to other agents. To resolve this problem, this paper proposes a novel framework to deal with delays as well as the non-stationary training issue of multi-agent tasks with model-free deep reinforcement learning. We formally define the Delay-Aware Markov Game that incorporates the delays of all agents in the environment. To solve Delay-Aware Markov Games, we apply centralized training and decentralized execution that allows agents to use extra information to ease the non-stationarity issue of the multi-agent systems during training, without the need of a centralized controller during execution. Experiments are conducted in multi-agent particle environments including cooperative communication, cooperative navigation, and competitive

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keychen2020delay

Related papers

• Reinforcement Learning Via Conservative Agent For Environments With Random Delays (2025)0.00
• DACOM: Learning Delay-aware Communication For Multi-agent Reinforcement Learning (2022)0.00
• Multi-agent Reinforcement Learning With Reward Delays (2022)0.00
• DSDF: An Approach To Handle Stochastic Agents In Collaborative Multi-agent Reinforcement Learning (2021)0.00
• Deep Multiagent Reinforcement Learning: Challenges And Directions (2021)0.00
• Revisiting State Augmentation Methods For Reinforcement Learning With Stochastic Delays (2021)10.35
• Research On Multi-agent Communication And Collaborative Decision-making Based On Deep Reinforcement Learning (2023)0.00
• Decomposing Communication Gain And Delay Cost Under Cross-timestep Delays In Cooperative Multi-agent Reinforcement Learning (2026)0.00