Adaptive Surrogate Gradients For Sequential Reinforcement Learning In Spiking Neural Networks

Abstract

Neuromorphic computing systems are set to revolutionize energy-constrained robotics by achieving orders-of-magnitude efficiency gains, while enabling native temporal processing. Spiking Neural Networks (SNNs) represent a promising algorithmic approach for these systems, yet their application to complex control tasks faces two critical challenges: (1) the non-differentiable nature of spiking neurons necessitates surrogate gradients with unclear optimization properties, and (2) the stateful dynamics of SNNs require training on sequences, which in reinforcement learning (RL) is hindered by limited sequence lengths during early training, preventing the network from bridging its warm-up period. We address these challenges by systematically analyzing surrogate gradient slope settings, showing that shallower slopes increase gradient magnitude in deeper layers but reduce alignment with true gradients. In supervised learning, we find no clear preference for fixed or scheduled slopes. The effe

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyberghe2025adaptive

Related papers

• Fully Spiking Actor Network With Intra-layer Connections For Reinforcement Learning (2024)0.00
• Deep Reinforcement Learning With Spiking Q-learning (2022)0.00
• Learning First-to-spike Policies For Neuromorphic Control Using Policy Gradients (2018)8.60
• Learning To Control Dynamical Agents Via Spiking Neural Networks And Metropolis-hastings Sampling (2025)0.00
• Trapezoidal Gradient Descent For Effective Reinforcement Learning In Spiking Networks (2024)0.00
• A Low Latency Adaptive Coding Spiking Framework For Deep Reinforcement Learning (2022)7.16
• Learning Fast Changing Slow In Spiking Neural Networks (2024)3.58
• GRSN: Gated Recurrent Spiking Neurons For Pomdps And MARL (2024)2.26