Influence-aware Memory Architectures For Deep Reinforcement Learning

Abstract

Due to its perceptual limitations, an agent may have too little information about the state of the environment to act optimally. In such cases, it is important to keep track of the observation history to uncover hidden state. Recent deep reinforcement learning methods use recurrent neural networks (RNN) to memorize past observations. However, these models are expensive to train and have convergence difficulties, especially when dealing with high dimensional input spaces. In this paper, we propose influence-aware memory (IAM), a theoretically inspired memory architecture that tries to alleviate the training difficulties by restricting the input of the recurrent layers to those variables that influence the hidden state information. Moreover, as opposed to standard RNNs, in which every piece of information used for estimating Q values is inevitably fed back into the network for the next prediction, our model allows information to flow without being necessarily stored in the RNN's internal

Stats

• citations1
• S2 citations—
• github stars0
• HF likes0
• heat score2.26
• arxiv keysuau2019influence

Related papers

• Stable Hadamard Memory: Revitalizing Memory-augmented Agents For Reinforcement Learning (2024)0.00
• Low-pass Recurrent Neural Networks - A Memory Architecture For Longer-term Correlation Discovery (2018)0.00
• Perception-prediction-reaction Agents For Deep Reinforcement Learning (2020)0.00
• Reservoir Computing For Fast, Simplified Reinforcement Learning On Memory Tasks (2024)0.00
• Dynamic Deep-reinforcement-learning Algorithm In Partially Observable Markov Decision Processes (2023)0.00
• Augmented Replay Memory In Reinforcement Learning With Continuous Control (2019)5.24
• Memory Lens: How Much Memory Does An Agent Use? (2016)0.00
• Map-based Experience Replay: A Memory-efficient Solution To Catastrophic Forgetting In Reinforcement Learning (2023)9.23