Riskq: Risk-sensitive Multi-agent Reinforcement Learning Value Factorization
2023 · Siqi Shen, Chennan Ma, Chao Li, et al.
Abstract
Multi-agent systems are characterized by environmental uncertainty, varying policies of agents, and partial observability, which result in significant risks. In the context of Multi-Agent Reinforcement Learning (MARL), learning coordinated and decentralized policies that are sensitive to risk is challenging. To formulate the coordination requirements in risk-sensitive MARL, we introduce the Risk-sensitive Individual-Global-Max (RIGM) principle as a generalization of the Individual-Global-Max (IGM) and Distributional IGM (DIGM) principles. This principle requires that the collection of risk-sensitive action selections of each agent should be equivalent to the risk-sensitive action selection of the central policy. Current MARL value factorization methods do not satisfy the RIGM principle for common risk metrics such as the Value at Risk (VaR) metric or distorted risk measurements. Therefore, we propose RiskQ to address this limitation, which models the joint return distribution by modeli
Authors
(none)
Tags
Stats
Related papers
- Residual Q-networks For Value Function Factorizing In Multi-agent Reinforcement Learning (2022)10.21
- RMIX: Learning Risk-sensitive Policies For Cooperative Reinforcement Learning Agents (2021)0.00
- Qfree: A Universal Value Function Factorization For Multi-agent Reinforcement Learning (2023)0.00
- QR-MIX: Distributional Value Function Factorisation For Cooperative Multi-agent Reinforcement Learning (2020)0.00
- DFAC Framework: Factorizing The Value Function Via Quantile Mixture For Multi-agent Distributional Q-learning (2021)0.00
- Risk-aware Distributed Multi-agent Reinforcement Learning (2023)3.58
- MCMARL: Parameterizing Value Function Via Mixture Of Categorical Distributions For Multi-agent Reinforcement Learning (2022)0.00
- Risk-sensitive Multi-agent Reinforcement Learning In Network Aggregative Markov Games (2024)0.00