Visual Explanation Using Attention Mechanism In Actor-critic-based Deep Reinforcement Learning

Abstract

Deep reinforcement learning (DRL) has great potential for acquiring the optimal action in complex environments such as games and robot control. However, it is difficult to analyze the decision-making of the agent, i.e., the reasons it selects the action acquired by learning. In this work, we propose Mask-Attention A3C (Mask A3C), which introduces an attention mechanism into Asynchronous Advantage Actor-Critic (A3C), which is an actor-critic-based DRL method, and can analyze the decision-making of an agent in DRL. A3C consists of a feature extractor that extracts features from an image, a policy branch that outputs the policy, and a value branch that outputs the state value. In this method, we focus on the policy and value branches and introduce an attention mechanism into them. The attention mechanism applies a mask processing to the feature maps of each branch using mask-attention that expresses the judgment reason for the policy and state value with a heat map. We visualized mask-att

Stats

• citations22
• S2 citations—
• github stars0
• HF likes0
• heat score10.21
• arxiv keyitaya2021visual

Related papers

• Why The Agent Made That Decision: Contrastive Explanation Learning For Reinforcement Learning (2024)0.00
• Machine Versus Human Attention In Deep Reinforcement Learning Tasks (2020)0.00
• Learning To Identify Critical States For Reinforcement Learning From Videos (2023)8.76
• Actor-attention-critic For Multi-agent Reinforcement Learning (2018)0.00
• Broad Critic Deep Actor Reinforcement Learning For Continuous Control (2024)0.00
• Attention Actor-critic Algorithm For Multi-agent Constrained Co-operative Reinforcement Learning (2021)0.00
• MAGIC-MASK: Multi-agent Guided Inter-agent Collaboration With Mask-based Explainability For Reinforcement Learning (2025)0.00
• Interpretability For Conditional Coordinated Behavior In Multi-agent Reinforcement Learning (2023)3.58