Risk-aware Distributed Multi-agent Reinforcement Learning

Abstract

Autonomous cyber and cyber-physical systems need to perform decision-making, learning, and control in unknown environments. Such decision-making can be sensitive to multiple factors, including modeling errors, changes in costs, and impacts of events in the tails of probability distributions. Although multi-agent reinforcement learning (MARL) provides a framework for learning behaviors through repeated interactions with the environment by minimizing an average cost, it will not be adequate to overcome the above challenges. In this paper, we develop a distributed MARL approach to solve decision-making problems in unknown environments by learning risk-aware actions. We use the conditional value-at-risk (CVaR) to characterize the cost function that is being minimized, and define a Bellman operator to characterize the value function associated to a given state-action pair. We prove that this operator satisfies a contraction property, and that it converges to the optimal value function. We t

Stats

• citations2
• S2 citations—
• github stars0
• HF likes0
• heat score3.58
• arxiv keymaruf2023risk

Related papers

• Risk-sensitive Multi-agent Reinforcement Learning In Network Aggregative Markov Games (2024)0.00
• RMIX: Learning Risk-sensitive Policies For Cooperative Reinforcement Learning Agents (2021)0.00
• Toward Risk-based Optimistic Exploration For Cooperative Multi-agent Reinforcement Learning (2023)0.00
• Safe Multi-agent Reinforcement Learning With Convergence To Generalized Nash Equilibrium (2024)0.00
• Tackling Uncertainties In Multi-agent Reinforcement Learning Through Integration Of Agent Termination Dynamics (2025)2.26
• Multi-agent Reinforcement Learning In Stochastic Networked Systems (2020)0.00
• Fully Decentralized Multi-agent Reinforcement Learning With Networked Agents (2018)0.00
• CAMMARL: Conformal Action Modeling In Multi Agent Reinforcement Learning (2023)0.00