A Unified Framework For Factorizing Distributional Value Functions For Multi-agent Reinforcement Learning

Abstract

In fully cooperative multi-agent reinforcement learning (MARL) settings, environments are highly stochastic due to the partial observability of each agent and the continuously changing policies of other agents. To address the above issues, we proposed a unified framework, called DFAC, for integrating distributional RL with value function factorization methods. This framework generalizes expected value function factorization methods to enable the factorization of return distributions. To validate DFAC, we first demonstrate its ability to factorize the value functions of a simple matrix game with stochastic rewards. Then, we perform experiments on all Super Hard maps of the StarCraft Multi-Agent Challenge and six self-designed Ultra Hard maps, showing that DFAC is able to outperform a number of baselines.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keysun2023a

Related papers

• DFAC Framework: Factorizing The Value Function Via Quantile Mixture For Multi-agent Distributional Q-learning (2021)0.00
• Factored Value Functions For Graph-based Multi-agent Reinforcement Learning (2026)0.00
• Qfree: A Universal Value Function Factorization For Multi-agent Reinforcement Learning (2023)0.00
• PAC: Assisted Value Factorisation With Counterfactual Predictions In Multi-agent Reinforcement Learning (2022)0.00
• Towards Understanding Cooperative Multi-agent Q-learning With Value Factorization (2020)0.00
• MCMARL: Parameterizing Value Function Via Mixture Of Categorical Distributions For Multi-agent Reinforcement Learning (2022)0.00
• More Centralized Training, Still Decentralized Execution: Multi-agent Conditional Policy Factorization (2022)0.00
• Beyond Monotonicity: Revisiting Factorization Principles In Multi-agent Q-learning (2025)0.00