Continuous Episodic Control

Abstract

Non-parametric episodic memory can be used to quickly latch onto high-rewarded experience in reinforcement learning tasks. In contrast to parametric deep reinforcement learning approaches in which reward signals need to be back-propagated slowly, these methods only need to discover the solution once, and may then repeatedly solve the task. However, episodic control solutions are stored in discrete tables, and this approach has so far only been applied to discrete action space problems. Therefore, this paper introduces Continuous Episodic Control (CEC), a novel non-parametric episodic memory algorithm for sequential decision making in problems with a continuous action space. Results on several sparse-reward continuous control environments show that our proposed method learns faster than state-of-the-art model-free RL and memory-augmented RL algorithms, while maintaining good long-run performance as well. In short, CEC can be a fast approach for learning in continuous control tasks.

Stats

• citations1
• S2 citations—
• github stars0
• HF likes0
• heat score2.26
• arxiv keyyang2022continuous

Related papers

• Sequential Memory Improves Sample And Memory Efficiency In Episodic Control (2021)0.00
• Memory-efficient Episodic Control Reinforcement Learning With Dynamic Online K-means (2019)0.00
• Sample-efficient Reinforcement Learning With Maximum Entropy Mellowmax Episodic Control (2019)0.00
• Episodic Reinforcement Learning With Expanded State-reward Space (2024)0.00
• Is Prioritized Sweeping The Better Episodic Control? (2017)0.00
• Centralized Cooperative Exploration Policy For Continuous Control Tasks (2023)0.00
• Asynchronous Episodic Deep Deterministic Policy Gradient: Towards Continuous Control In Computationally Complex Environments (2019)0.00
• Adaptive Discretization For Episodic Reinforcement Learning In Metric Spaces (2019)2.26