Unraveling The Complexity Of Memory In RL Agents: An Approach For Classification And Evaluation

Abstract

The incorporation of memory into agents is essential for numerous tasks within the domain of Reinforcement Learning (RL). In particular, memory is paramount for tasks that require the use of past information, adaptation to novel environments, and improved sample efficiency. However, the term "memory" encompasses a wide range of concepts, which, coupled with the lack of a unified methodology for validating an agent's memory, leads to erroneous judgments about agents' memory capabilities and prevents objective comparison with other memory-enhanced agents. This paper aims to streamline the concept of memory in RL by providing practical precise definitions of agent memory types, such as long-term vs. short-term memory and declarative vs. procedural memory, inspired by cognitive science. Using these definitions, we categorize different classes of agent memory, propose a robust experimental methodology for evaluating the memory capabilities of RL agents, and standardize evaluations. Furtherm

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keycherepanov2024unraveling

Related papers

• Generalization Of Reinforcement Learners With Working And Episodic Memory (2019)0.00
• The Act Of Remembering: A Study In Partially Observable Reinforcement Learning (2020)0.00
• Stable Hadamard Memory: Revitalizing Memory-augmented Agents For Reinforcement Learning (2024)0.00
• Unsupervised Predictive Memory In A Goal-directed Agent (2018)0.00
• Adamemento: Adaptive Memory-assisted Policy Optimization For Reinforcement Learning (2024)0.00
• Memory Lens: How Much Memory Does An Agent Use? (2016)0.00
• Stabilising Experience Replay For Deep Multi-agent Reinforcement Learning (2017)0.00
• Towards Mental Time Travel: A Hierarchical Memory For Reinforcement Learning Agents (2021)0.00