Local And Global Explanations Of Agent Behavior: Integrating Strategy Summaries With Saliency Maps

Abstract

With advances in reinforcement learning (RL), agents are now being developed in high-stakes application domains such as healthcare and transportation. Explaining the behavior of these agents is challenging, as the environments in which they act have large state spaces, and their decision-making can be affected by delayed rewards, making it difficult to analyze their behavior. To address this problem, several approaches have been developed. Some approaches attempt to convey the \(\textit\{global\}\) behavior of the agent, describing the actions it takes in different states. Other approaches devised \(\textit\{local\}\) explanations which provide information regarding the agent's decision-making in a particular state. In this paper, we combine global and local explanation methods, and evaluate their joint and separate contributions, providing (to the best of our knowledge) the first user study of combined local and global explanations for RL agents. Specifically, we augment strategy summ

Stats

• citations37
• S2 citations—
• github stars0
• HF likes0
• heat score11.85
• arxiv keyhuber2020local

Related papers

• Integrating Policy Summaries With Reward Decomposition For Explaining Reinforcement Learning Agents (2022)7.16
• Explain Your Move: Understanding Agent Actions Using Specific And Relevant Feature Attribution (2019)0.00
• Exploratory Not Explanatory: Counterfactual Analysis Of Saliency Maps For Deep Reinforcement Learning (2019)0.00
• Ganterfactual-rl: Understanding Reinforcement Learning Agents' Strategies Through Visual Counterfactual Explanations (2023)2.26
• Visualizing And Understanding Atari Agents (2017)0.00
• Why The Agent Made That Decision: Contrastive Explanation Learning For Reinforcement Learning (2024)0.00
• Talktoagent: A Human-centric Explanation Of Reinforcement Learning Agents With Large Language Models (2025)0.00
• What Did You Think Would Happen? Explaining Agent Behaviour Through Intended Outcomes (2020)0.00