Multi-agent Fully Decentralized Value Function Learning With Linear Convergence Rates
2018 · Lucas Cassano, Kun Yuan, Ali H. Sayed
Abstract
This work develops a fully decentralized multi-agent algorithm for policy evaluation. The proposed scheme can be applied to two distinct scenarios. In the first scenario, a collection of agents have distinct datasets gathered following different behavior policies (none of which is required to explore the full state space) in different instances of the same environment and they all collaborate to evaluate a common target policy. The network approach allows for efficient exploration of the state space and allows all agents to converge to the optimal solution even in situations where neither agent can converge on its own without cooperation. The second scenario is that of multi-agent games, in which the state is global and rewards are local. In this scenario, agents collaborate to estimate the value function of a target team policy. The proposed algorithm combines off-policy learning, eligibility traces and linear function approximation. The proposed algorithm is of the variance-reduced k
Authors
(none)
Tags
Stats
Related papers
- Distributed Value Function Approximation For Collaborative Multi-agent Reinforcement Learning (2020)8.60
- Offline Decentralized Multi-agent Reinforcement Learning (2021)7.50
- Finite-sample Analysis Of Decentralized Temporal-difference Learning With Linear Function Approximation (2019)0.00
- Towards Understanding Cooperative Multi-agent Q-learning With Value Factorization (2020)0.00
- Breaking The Curse Of Multiagency: Provably Efficient Decentralized Multi-agent RL With Function Approximation (2023)0.00
- Value Propagation For Decentralized Networked Deep Multi-agent Reinforcement Learning (2019)0.00
- More Centralized Training, Still Decentralized Execution: Multi-agent Conditional Policy Factorization (2022)0.00
- Monotonic Value Function Factorisation For Deep Multi-agent Reinforcement Learning (2020)0.00