Efficient Open-world Reinforcement Learning Via Knowledge Distillation And Autonomous Rule Discovery

Abstract

Deep reinforcement learning suffers from catastrophic forgetting and sample inefficiency making it less applicable to the ever-changing real world. However, the ability to use previously learned knowledge is essential for AI agents to quickly adapt to novelties. Often, certain spatial information observed by the agent in the previous interactions can be leveraged to infer task-specific rules. Inferred rules can then help the agent to avoid potentially dangerous situations in the previously unseen states and guide the learning process increasing agent's novelty adaptation speed. In this work, we propose a general framework that is applicable to deep reinforcement learning agents. Our framework provides the agent with an autonomous way to discover the task-specific rules in the novel environments and self-supervise it's learning. We provide a rule-driven deep Q-learning agent (RDQ) as one possible implementation of that framework. We show that RDQ successfully extracts task-specific rule

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keynikonova2023efficient

Related papers

• Knowru: Knowledge Reusing Via Knowledge Distillation In Multi-agent Reinforcement Learning (2021)9.23
• Automaton Distillation: Neuro-symbolic Transfer Learning For Deep Reinforcement Learning (2023)0.00
• KD-MARL: Resource-aware Knowledge Distillation In Multi-agent Reinforcement Learning (2026)0.00
• Improving Generalization To New Environments And Removing Catastrophic Forgetting In Reinforcement Learning By Using An Eco-system Of Agents (2022)0.00
• Goal-conditioned Q-learning As Knowledge Distillation (2022)3.58
• A New Framework For Multi-agent Reinforcement Learning -- Centralized Training And Exploration With Decentralized Execution Via Policy Distillation (2019)0.00
• Interactive Reinforcement Learning With Dynamic Reuse Of Prior Knowledge From Human/agent's Demonstration (2018)8.60
• Periodic Intra-ensemble Knowledge Distillation For Reinforcement Learning (2020)4.52