Generation Of Policy-level Explanations For Reinforcement Learning

Abstract

Though reinforcement learning has greatly benefited from the incorporation of neural networks, the inability to verify the correctness of such systems limits their use. Current work in explainable deep learning focuses on explaining only a single decision in terms of input features, making it unsuitable for explaining a sequence of decisions. To address this need, we introduce Abstracted Policy Graphs, which are Markov chains of abstract states. This representation concisely summarizes a policy so that individual decisions can be explained in the context of expected future transitions. Additionally, we propose a method to generate these Abstracted Policy Graphs for deterministic policies given a learned value function and a set of observed transitions, potentially off-policy transitions used during training. Since no restrictions are placed on how the value function is generated, our method is compatible with many existing reinforcement learning methods. We prove that the worst-case ti

Stats

• citations32
• S2 citations—
• github stars0
• HF likes0
• heat score11.39
• arxiv keytopin2019generation

Related papers

• Experiential Explanations For Reinforcement Learning (2022)2.26
• From Explainability To Interpretability: Interpretable Policies In Reinforcement Learning Via Model Explanation (2025)0.00
• End-to-end Neuro-symbolic Reinforcement Learning With Textual Explanations (2024)0.00
• S-REINFORCE: A Neuro-symbolic Policy Gradient Approach For Interpretable Reinforcement Learning (2023)0.00
• Counterfactual State Explanations For Reinforcement Learning Agents Via Generative Deep Learning (2021)13.23
• "so, Tell Me About Your Policy...": Distillation Of Interpretable Policies From Deep Reinforcement Learning Agents (2025)0.00
• Learning Impartial Policies For Sequential Counterfactual Explanations Using Deep Reinforcement Learning (2023)0.00
• Generating Explanations From Deep Reinforcement Learning Using Episodic Memory (2022)0.00