Is Prioritized Sweeping The Better Episodic Control?

Abstract

Episodic control has been proposed as a third approach to reinforcement learning, besides model-free and model-based control, by analogy with the three types of human memory. i.e. episodic, procedural and semantic memory. But the theoretical properties of episodic control are not well investigated. Here I show that in deterministic tree Markov decision processes, episodic control is equivalent to a form of prioritized sweeping in terms of sample efficiency as well as memory and computation demands. For general deterministic and stochastic environments, prioritized sweeping performs better even when memory and computation demands are restricted to be equal to those of episodic control. These results suggest generalizations of prioritized sweeping to partially observable environments, its combined use with function approximation and the search for possible implementations of prioritized sweeping in brains.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keybrea2017is

Related papers

• Continuous Episodic Control (2022)2.26
• Sequential Memory Improves Sample And Memory Efficiency In Episodic Control (2021)0.00
• Memory-efficient Episodic Control Reinforcement Learning With Dynamic Online K-means (2019)0.00
• Prioritized Sweeping Neural Dynaq With Multiple Predecessors, And Hippocampal Replays (2018)6.34
• Sample-efficient Reinforcement Learning With Maximum Entropy Mellowmax Episodic Control (2019)0.00
• Successor Feature Neural Episodic Control (2021)0.00
• Towards Sample-efficient Episodic Control With DAC-ML (2020)0.00
• Asynchronous Episodic Deep Deterministic Policy Gradient: Towards Continuous Control In Computationally Complex Environments (2019)0.00