Weighted Double Deep Multiagent Reinforcement Learning In Stochastic Cooperative Environments

Abstract

Recently, multiagent deep reinforcement learning (DRL) has received increasingly wide attention. Existing multiagent DRL algorithms are inefficient when facing with the non-stationarity due to agents update their policies simultaneously in stochastic cooperative environments. This paper extends the recently proposed weighted double estimator to the multiagent domain and propose a multiagent DRL framework, named weighted double deep Q-network (WDDQN). By utilizing the weighted double estimator and the deep neural network, WDDQN can not only reduce the bias effectively but also be extended to scenarios with raw visual inputs. To achieve efficient cooperation in the multiagent domain, we introduce the lenient reward network and the scheduled replay strategy. Experiments show that the WDDQN outperforms the existing DRL and multiaent DRL algorithms, i.e., double DQN and lenient Q-learning, in terms of the average reward and the convergence rate in stochastic cooperative environments.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzheng2018weighted

Related papers

• Deep Reinforcement Learning With Weighted Q-learning (2020)0.00
• Lenient Multi-agent Deep Reinforcement Learning (2017)4.52
• Distributional Reward Estimation For Effective Multi-agent Deep Reinforcement Learning (2022)0.00
• Deep Multi-agent Reinforcement Learning With Discrete-continuous Hybrid Action Spaces (2019)12.47
• Hierarchical Reinforcement Learning With Opponent Modeling For Distributed Multi-agent Cooperation (2022)5.84
• A New Framework For Multi-agent Reinforcement Learning -- Centralized Training And Exploration With Decentralized Execution Via Policy Distillation (2019)0.00
• DSDF: An Approach To Handle Stochastic Agents In Collaborative Multi-agent Reinforcement Learning (2021)0.00
• Directionality Reinforcement Learning To Operate Multi-agent System Without Communication (2021)0.00