Learning Of Generalizable And Interpretable Knowledge In Grid-based Reinforcement Learning Environments

Abstract

Understanding the interactions of agents trained with deep reinforcement learning is crucial for deploying agents in games or the real world. In the former, unreasonable actions confuse players. In the latter, that effect is even more significant, as unexpected behavior cause accidents with potentially grave and long-lasting consequences for the involved individuals. In this work, we propose using program synthesis to imitate reinforcement learning policies after seeing a trajectory of the action sequence. Programs have the advantage that they are inherently interpretable and verifiable for correctness. We adapt the state-of-the-art program synthesis system DreamCoder for learning concepts in grid-based environments, specifically, a navigation task and two miniature versions of Atari games, Space Invaders and Asterix. By inspecting the generated libraries, we can make inferences about the concepts the black-box agent has learned and better understand the agent's behavior. We achieve th

Stats

• citations2
• S2 citations—
• github stars0
• HF likes0
• heat score3.58
• arxiv keyeberhardinger2023learning

Related papers

• Programmatic Reinforcement Learning: Navigating Gridworlds (2024)0.00
• Learning What To Do By Simulating The Past (2021)0.00
• Illuminating Generalization In Deep Reinforcement Learning Through Procedural Level Generation (2018)0.00
• Visual Transfer Between Atari Games Using Competitive Reinforcement Learning (2018)7.50
• Interpretable Policies For Reinforcement Learning By Genetic Programming (2017)14.76
• Forward-backward Reinforcement Learning (2018)0.00
• Do Agents Dream Of Electric Sheep?: Improving Generalization In Reinforcement Learning Through Generative Learning (2024)0.00
• Knowledge Transfer In Deep Reinforcement Learning Via An Rl-specific Gan-based Correspondence Function (2022)2.26