Learning "what-if" Explanations For Sequential Decision-making

Abstract

Building interpretable parameterizations of real-world decision-making on the basis of demonstrated behavior -- i.e. trajectories of observations and actions made by an expert maximizing some unknown reward function -- is essential for introspecting and auditing policies in different institutions. In this paper, we propose learning explanations of expert decisions by modeling their reward function in terms of preferences with respect to "what if" outcomes: Given the current history of observations, what would happen if we took a particular action? To learn these cost-benefit tradeoffs associated with the expert's actions, we integrate counterfactual reasoning into batch inverse reinforcement learning. This offers a principled way of defining reward functions and explaining expert behavior, and also satisfies the constraints of real-world decision-making -- where active experimentation is often impossible (e.g. in healthcare). Additionally, by estimating the effects of different actions

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keybica2020learning

Related papers

• Inverse Decision Modeling: Learning Interpretable Representations Of Behavior (2023)0.00
• Experiential Explanations For Reinforcement Learning (2022)2.26
• Learning Impartial Policies For Sequential Counterfactual Explanations Using Deep Reinforcement Learning (2023)0.00
• Reward Design For Justifiable Sequential Decision-making (2024)0.00
• Explaining Learned Reward Functions With Counterfactual Trajectories (2024)0.00
• Integrating Policy Summaries With Reward Decomposition For Explaining Reinforcement Learning Agents (2022)7.16
• Explaining RL Decisions With Trajectories (2023)0.00
• Explaining Reinforcement Learning Policies Through Counterfactual Trajectories (2022)0.00