Efficient Off-policy Reinforcement Learning Via Brain-inspired Computing

Abstract

Reinforcement Learning (RL) has opened up new opportunities to enhance existing smart systems that generally include a complex decision-making process. However, modern RL algorithms, e.g., Deep Q-Networks (DQN), are based on deep neural networks, resulting in high computational costs. In this paper, we propose QHD, an off-policy value-based Hyperdimensional Reinforcement Learning, that mimics brain properties toward robust and real-time learning. QHD relies on a lightweight brain-inspired model to learn an optimal policy in an unknown environment. On both desktop and power-limited embedded platforms, QHD achieves significantly better overall efficiency than DQN while providing higher or comparable rewards. QHD is also suitable for highly-efficient reinforcement learning with great potential for online and real-time learning. Our solution supports a small experience replay batch size that provides 12.3 times speedup compared to DQN while ensuring minimal quality loss. Our evaluation sho

Stats

• citations12
• S2 citations—
• github stars0
• HF likes0
• heat score8.35
• arxiv keyni2022efficient

Related papers

• Efficient Deep Reinforcement Learning With Predictive Processing Proximal Policy Optimization (2022)0.00
• Reinforcement Learning With Brain-inspired Modulation Can Improve Adaptation To Environmental Changes (2022)0.00
• Accelerated Methods For Deep Reinforcement Learning (2018)0.00
• Human-inspired Framework To Accelerate Reinforcement Learning (2023)0.00
• Deep Q-networks For Accelerating The Training Of Deep Neural Networks (2016)0.00
• Data Efficient Training For Reinforcement Learning With Adaptive Behavior Policy Sharing (2020)0.00
• Hybrid RL: Using Both Offline And Online Data Can Make RL Efficient (2022)0.00
• Deep Reinforcement Learning With Spiking Q-learning (2022)0.00