Talktoagent: A Human-centric Explanation Of Reinforcement Learning Agents With Large Language Models

Abstract

Explainable Reinforcement Learning (XRL) has emerged as a promising approach in improving the transparency of Reinforcement Learning (RL) agents. However, there remains a gap between complex RL policies and domain experts, due to the limited comprehensibility of XRL results and isolated coverage of current XRL approaches that leave users uncertain about which tools to employ. To address these challenges, we introduce TalkToAgent, a multi-agent Large Language Models (LLM) framework that delivers interactive, natural language explanations for RL policies. The architecture with five specialized LLM agents (Coordinator, Explainer, Coder, Evaluator, and Debugger) enables TalkToAgent to automatically map user queries to relevant XRL tools and clarify an agent's actions in terms of either key state variables, expected outcomes, or counterfactual explanations. Moreover, our approach extends previous counterfactual explanations by deriving alternative scenarios from qualitative behavioral descr

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keykim2025talktoagent

Related papers

• Why The Agent Made That Decision: Contrastive Explanation Learning For Reinforcement Learning (2024)0.00
• Mental Modeling Of Reinforcement Learning Agents By Language Models (2024)0.00
• Explainable Reinforcement Learning Agents Using World Models (2025)0.00
• A Survey Of Explainable Reinforcement Learning: Targets, Methods And Needs (2025)0.00
• Explainability In Deep Reinforcement Learning (2020)0.00
• A Survey On Explainable Reinforcement Learning: Concepts, Algorithms, Challenges (2022)0.00
• Explaining Reinforcement Learning Agents Through Counterfactual Action Outcomes (2023)5.84
• Interestingness Elements For Explainable Reinforcement Learning: Understanding Agents' Capabilities And Limitations (2019)14.55