Multi-agent Determinantal Q-learning

Abstract

Centralized training with decentralized execution has become an important paradigm in multi-agent learning. Though practical, current methods rely on restrictive assumptions to decompose the centralized value function across agents for execution. In this paper, we eliminate this restriction by proposing multi-agent determinantal Q-learning. Our method is established on Q-DPP, an extension of determinantal point process (DPP) with partition-matroid constraint to multi-agent setting. Q-DPP promotes agents to acquire diverse behavioral models; this allows a natural factorization of the joint Q-functions with no need for *a priori* structural constraints on the value function or special network architectures. We demonstrate that Q-DPP generalizes major solutions including VDN, QMIX, and QTRAN on decentralizable cooperative tasks. To efficiently draw samples from Q-DPP, we adopt an existing sample-by-projection sampler with theoretical approximation guarantee. The sampler also benefits expl

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyyang2020multi

Related papers

• Q-value Path Decomposition For Deep Multiagent Reinforcement Learning (2020)0.00
• Factorized Q-learning For Large-scale Multi-agent Systems (2018)11.58
• Qatten: A General Framework For Cooperative Multiagent Reinforcement Learning (2020)0.00
• Decentralised Q-learning For Multi-agent Markov Decision Processes With A Satisfiability Criterion (2023)0.00
• MA2QL: A Minimalist Approach To Fully Decentralized Multi-agent Reinforcement Learning (2022)0.00
• PTDE: Personalized Training With Distilled Execution For Multi-agent Reinforcement Learning (2022)0.00
• Provably Efficient Multi-agent Reinforcement Learning With Fully Decentralized Communication (2021)0.00
• Mitigating Relative Over-generalization In Multi-agent Reinforcement Learning (2024)0.00