Learning To Communicate Using Counterfactual Reasoning

Abstract

Learning to communicate in order to share state information is an active problem in the area of multi-agent reinforcement learning (MARL). The credit assignment problem, the non-stationarity of the communication environment and the creation of influenceable agents are major challenges within this research field which need to be overcome in order to learn a valid communication protocol. This paper introduces the novel multi-agent counterfactual communication learning (MACC) method which adapts counterfactual reasoning in order to overcome the credit assignment problem for communicating agents. Secondly, the non-stationarity of the communication environment while learning the communication Q-function is overcome by creating the communication Q-function using the action policy of the other agents and the Q-function of the action environment. Additionally, a social loss function is introduced in order to create influenceable agents which is required to learn a valid communication protocol.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyvanneste2020learning

Related papers

• AC2C: Adaptively Controlled Two-hop Communication For Multi-agent Reinforcement Learning (2023)0.00
• Counterfactual Multi-agent Reinforcement Learning With Graph Convolution Communication (2020)0.00
• Learning Practical Communication Strategies In Cooperative Multi-agent Reinforcement Learning (2022)0.00
• Cooperative Multi-agent RL With Communication Constraints (2026)0.00
• Social Influence As Intrinsic Motivation For Multi-agent Deep Reinforcement Learning (2018)0.00
• On The Role Of Emergent Communication For Social Learning In Multi-agent Reinforcement Learning (2023)0.00
• MACCA: Offline Multi-agent Reinforcement Learning With Causal Credit Assignment (2023)0.00
• Context-aware Communication For Multi-agent Reinforcement Learning (2023)3.14