QMIX: Monotonic Value Function Factorisation For Deep Multi-agent Reinforcement Learning

Abstract

In many real-world settings, a team of agents must coordinate their behaviour while acting in a decentralised way. At the same time, it is often possible to train the agents in a centralised fashion in a simulated or laboratory setting, where global state information is available and communication constraints are lifted. Learning joint action-values conditioned on extra state information is an attractive way to exploit centralised learning, but the best strategy for then extracting decentralised policies is unclear. Our solution is QMIX, a novel value-based method that can train decentralised policies in a centralised end-to-end fashion. QMIX employs a network that estimates joint action-values as a complex non-linear combination of per-agent values that condition only on local observations. We structurally enforce that the joint-action value is monotonic in the per-agent values, which allows tractable maximisation of the joint action-value in off-policy learning, and guarantees consis

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyrashid2018qmix

Related papers

• Monotonic Value Function Factorisation For Deep Multi-agent Reinforcement Learning (2020)0.00
• Weighted QMIX: Expanding Monotonic Value Function Factorisation For Deep Multi-agent Reinforcement Learning (2020)0.00
• NQMIX: Non-monotonic Value Function Factorization For Deep Multi-agent Reinforcement Learning (2021)0.00
• POWQMIX: Weighted Value Factorization With Potentially Optimal Joint Actions Recognition For Cooperative Multi-agent Reinforcement Learning (2024)0.00
• QR-MIX: Distributional Value Function Factorisation For Cooperative Multi-agent Reinforcement Learning (2020)0.00
• MMD-MIX: Value Function Factorisation With Maximum Mean Discrepancy For Cooperative Multi-agent Reinforcement Learning (2021)0.00
• Concaveq: Non-monotonic Value Function Factorization Via Concave Representations In Deep Multi-agent Reinforcement Learning (2023)5.84
• Remix: Regret Minimization For Monotonic Value Function Factorization In Multiagent Reinforcement Learning (2023)0.00