Experiential Explanations For Reinforcement Learning

Abstract

Reinforcement learning (RL) systems can be complex and non-interpretable, making it challenging for non-AI experts to understand or intervene in their decisions. This is due in part to the sequential nature of RL in which actions are chosen because of their likelihood of obtaining future rewards. However, RL agents discard the qualitative features of their training, making it difficult to recover user-understandable information for "why" an action is chosen. We propose a technique Experiential Explanations to generate counterfactual explanations by training influence predictors along with the RL policy. Influence predictors are models that learn how different sources of reward affect the agent in different states, thus restoring information about how the policy reflects the environment. Two human evaluation studies revealed that participants presented with Experiential Explanations were better able to correctly guess what an agent would do than those presented with other standard types

Stats

• citations1
• S2 citations—
• github stars0
• HF likes0
• heat score2.26
• arxiv keyalabdulkarim2022experiential

Related papers

• Redefining Counterfactual Explanations For Reinforcement Learning: Overview, Challenges And Opportunities (2022)0.00
• Contrastive Explanations For Reinforcement Learning In Terms Of Expected Consequences (2018)0.00
• Why The Agent Made That Decision: Contrastive Explanation Learning For Reinforcement Learning (2024)0.00
• (when) Are Contrastive Explanations Of Reinforcement Learning Helpful? (2022)0.00
• Ganterfactual-rl: Understanding Reinforcement Learning Agents' Strategies Through Visual Counterfactual Explanations (2023)2.26
• Explainable Reinforcement Learning Via A Causal World Model (2023)9.03
• Explaining Reinforcement Learning Agents Through Counterfactual Action Outcomes (2023)5.84
• Causal State Distillation For Explainable Reinforcement Learning (2023)0.00