Competitive Multi-agent Deep Reinforcement Learning With Counterfactual Thinking

Abstract

Counterfactual thinking describes a psychological phenomenon that people re-infer the possible results with different solutions about things that have already happened. It helps people to gain more experience from mistakes and thus to perform better in similar future tasks. This paper investigates the counterfactual thinking for agents to find optimal decision-making strategies in multi-agent reinforcement learning environments. In particular, we propose a multi-agent deep reinforcement learning model with a structure which mimics the human-psychological counterfactual thinking process to improve the competitive abilities for agents. To this end, our model generates several possible actions (intent actions) with a parallel policy structure and estimates the rewards and regrets for these intent actions based on its current understanding of the environment. Our model incorporates a scenario-based framework to link the estimated regrets with its inner policies. During the iterations, our

Stats

• citations8
• S2 citations—
• github stars0
• HF likes0
• heat score7.16
• arxiv keywang2019competitive

Related papers

• Episodic Future Thinking Mechanism For Multi-agent Reinforcement Learning (2024)0.00
• Counterfactual States For Atari Agents Via Generative Deep Learning (2019)0.00
• Theory Of Mind As Intrinsic Motivation For Multi-agent Reinforcement Learning (2023)0.00
• Algorithms In Multi-agent Systems: A Holistic Perspective From Reinforcement Learning And Game Theory (2020)0.00
• ACTER: Diverse And Actionable Counterfactual Sequences For Explaining And Diagnosing RL Policies (2024)0.00
• Mingling Foresight With Imagination: Model-based Cooperative Multi-agent Reinforcement Learning (2022)0.00
• Probabilistic Recursive Reasoning For Multi-agent Reinforcement Learning (2019)0.00
• Ganterfactual-rl: Understanding Reinforcement Learning Agents' Strategies Through Visual Counterfactual Explanations (2023)2.26