Deep Reinforcement Learning With Spiking Q-learning

Abstract

With the help of special neuromorphic hardware, spiking neural networks (SNNs) are expected to realize artificial intelligence (AI) with less energy consumption. It provides a promising energy-efficient way for realistic control tasks by combining SNNs with deep reinforcement learning (RL). There are only a few existing SNN-based RL methods at present. Most of them either lack generalization ability or employ Artificial Neural Networks (ANNs) to estimate value function in training. The former needs to tune numerous hyper-parameters for each scenario, and the latter limits the application of different types of RL algorithm and ignores the large energy consumption in training. To develop a robust spike-based RL method, we draw inspiration from non-spiking interneurons found in insects and propose the deep spiking Q-network (DSQN), using the membrane voltage of non-spiking neurons as the representation of Q-value, which can directly learn robust policies from high-dimensional sensory inpu

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keychen2022deep

Related papers

• Human-level Control Through Directly-trained Deep Spiking Q-networks (2021)12.40
• Fully Spiking Actor Network With Intra-layer Connections For Reinforcement Learning (2024)0.00
• Learning First-to-spike Policies For Neuromorphic Control Using Policy Gradients (2018)8.60
• Hardware-aware Fine-tuning Of Spiking Q-networks On The Spinnaker2 Neuromorphic Platform (2025)0.00
• Adaptive Surrogate Gradients For Sequential Reinforcement Learning In Spiking Neural Networks (2025)0.00
• Distilling Neuron Spike With High Temperature In Reinforcement Learning Agents (2021)0.00
• A Low Latency Adaptive Coding Spiking Framework For Deep Reinforcement Learning (2022)7.16
• Learning To Control Dynamical Agents Via Spiking Neural Networks And Metropolis-hastings Sampling (2025)0.00