Deep Reinforcement Learning With Modulated Hebbian Plus Q Network Architecture

Abstract

This paper presents a new neural architecture that combines a modulated Hebbian network (MOHN) with DQN, which we call modulated Hebbian plus Q network architecture (MOHQA). The hypothesis is that such a combination allows MOHQA to solve difficult partially observable Markov decision process (POMDP) problems which impair temporal difference (TD)-based RL algorithms such as DQN, as the TD error cannot be easily derived from observations. The key idea is to use a Hebbian network with bio-inspired neural traces in order to bridge temporal delays between actions and rewards when confounding observations and sparse rewards result in inaccurate TD errors. In MOHQA, DQN learns low level features and control, while the MOHN contributes to the high-level decisions by associating rewards with past states and actions. Thus the proposed architecture combines two modules with significantly different learning algorithms, a Hebbian associative network and a classical DQN pipeline, exploiting the adva

Stats

• citations19
• S2 citations—
• github stars0
• HF likes0
• heat score9.76
• arxiv keyladosz2019deep

Related papers

• On Improving Deep Reinforcement Learning For Pomdps (2017)0.00
• Efficient Off-policy Reinforcement Learning Via Brain-inspired Computing (2022)8.35
• Dynamic Deep-reinforcement-learning Algorithm In Partially Observable Markov Decision Processes (2023)0.00
• Real-time Recurrent Reinforcement Learning (2023)2.26
• Modular Multi-objective Deep Reinforcement Learning With Decision Values (2017)10.74
• A Hybrid PAC Reinforcement Learning Algorithm (2020)0.00
• Classifying Options For Deep Reinforcement Learning (2016)0.00
• Towards Multi-agent Reinforcement Learning Using Quantum Boltzmann Machines (2021)0.00