Verifiable Reinforcement Learning Via Policy Extraction

Abstract

While deep reinforcement learning has successfully solved many challenging control tasks, its real-world applicability has been limited by the inability to ensure the safety of learned policies. We propose an approach to verifiable reinforcement learning by training decision tree policies, which can represent complex policies (since they are nonparametric), yet can be efficiently verified using existing techniques (since they are highly structured). The challenge is that decision tree policies are difficult to train. We propose VIPER, an algorithm that combines ideas from model compression and imitation learning to learn decision tree policies guided by a DNN policy (called the oracle) and its Q-function, and show that it substantially outperforms two baselines. We use VIPER to (i) learn a provably robust decision tree policy for a variant of Atari Pong with a symbolic state space, (ii) learn a decision tree policy for a toy game based on Pong that provably never loses, and (iii) learn

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keybastani2018verifiable

Related papers

• MAVIPER: Learning Decision Tree Policies For Interpretable Multi-agent Reinforcement Learning (2022)5.84
• "so, Tell Me About Your Policy...": Distillation Of Interpretable Policies From Deep Reinforcement Learning Agents (2025)0.00
• Fidelity-induced Interpretable Policy Extraction For Reinforcement Learning (2023)0.00
• Model-based Reinforcement Learning For Atari (2019)0.00
• Optimizing Interpretable Decision Tree Policies For Reinforcement Learning (2024)0.00
• Policy Improvement Reinforcement Learning (2026)0.00
• Mitigating Information Loss In Tree-based Reinforcement Learning Via Direct Optimization (2024)0.00
• Playing Atari With Six Neurons (2018)0.00