Asynchronous Credit Assignment For Multi-agent Reinforcement Learning

Abstract

Credit assignment is a critical problem in multi-agent reinforcement learning (MARL), aiming to identify agents' marginal contributions for optimizing cooperative policies. Current credit assignment methods typically assume synchronous decision-making among agents. However, many real-world scenarios require agents to act asynchronously without waiting for others. This asynchrony introduces conditional dependencies between actions, which pose great challenges to current methods. To address this issue, we propose an asynchronous credit assignment framework, incorporating a Virtual Synchrony Proxy (VSP) mechanism and a Multiplicative Value Decomposition (MVD) algorithm. VSP enables physically asynchronous actions to be virtually synchronized during credit assignment. We theoretically prove that VSP preserves both task equilibrium and algorithm convergence. Furthermore, MVD leverages multiplicative interactions to effectively model dependencies among asynchronous actions, offering theoreti

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyliang2024asynchronous

Related papers

• MACCA: Offline Multi-agent Reinforcement Learning With Causal Credit Assignment (2023)0.00
• Cooperative Game-theoretic Credit Assignment For Multi-agent Policy Gradients Via The Core (2025)0.00
• Assigning Credit With Partial Reward Decoupling In Multi-agent Proximal Policy Optimization (2024)0.00
• Challenges In Credit Assignment For Multi-agent Reinforcement Learning In Open Agent Systems (2025)0.00
• Shapley Counterfactual Credits For Multi-agent Reinforcement Learning (2021)12.40
• Adaptive Value Decomposition With Greedy Marginal Contribution Computation For Cooperative Multi-agent Reinforcement Learning (2023)3.58
• Learning Implicit Credit Assignment For Cooperative Multi-agent Reinforcement Learning (2020)0.00
• Cooperative Multi-agent Transfer Learning With Level-adaptive Credit Assignment (2021)0.00