Fully Spiking Actor Network With Intra-layer Connections For Reinforcement Learning

Abstract

With the help of special neuromorphic hardware, spiking neural networks (SNNs) are expected to realize artificial intelligence (AI) with less energy consumption. It provides a promising energy-efficient way for realistic control tasks by combining SNNs with deep reinforcement learning (DRL). In this paper, we focus on the task where the agent needs to learn multi-dimensional deterministic policies to control, which is very common in real scenarios. Recently, the surrogate gradient method has been utilized for training multi-layer SNNs, which allows SNNs to achieve comparable performance with the corresponding deep networks in this task. Most existing spike-based RL methods take the firing rate as the output of SNNs, and convert it to represent continuous action space (i.e., the deterministic policy) through a fully-connected (FC) layer. However, the decimal characteristic of the firing rate brings the floating-point matrix operations to the FC layer, making the whole SNN unable to depl

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keychen2024fully

Related papers

• Deep Reinforcement Learning With Spiking Q-learning (2022)0.00
• Adaptive Surrogate Gradients For Sequential Reinforcement Learning In Spiking Neural Networks (2025)0.00
• S2act: Simple Spiking Actor (2026)0.00
• Learning First-to-spike Policies For Neuromorphic Control Using Policy Gradients (2018)8.60
• Scalable Multi-task Learning Through Spiking Neural Networks With Adaptive Task-switching Policy For Intelligent Autonomous Agents (2025)0.00
• Human-level Control Through Directly-trained Deep Spiking Q-networks (2021)12.40
• A Low Latency Adaptive Coding Spiking Framework For Deep Reinforcement Learning (2022)7.16
• Population-coding And Dynamic-neurons Improved Spiking Actor Network For Reinforcement Learning (2021)0.00