Energy-efficient Deep Reinforcement Learning With Spiking Transformers

Abstract

Agent-based Transformers have been widely adopted in recent reinforcement learning advances due to their demonstrated ability to solve complex tasks. However, the high computational complexity of Transformers often results in significant energy consumption, limiting their deployment in real-world autonomous systems. Spiking neural networks (SNNs), with their biologically inspired structure, offer an energy-efficient alternative for machine learning. In this paper, a novel Spike-Transformer Reinforcement Learning (STRL) algorithm that combines the energy efficiency of SNNs with the powerful decision-making capabilities of reinforcement learning is developed. Specifically, an SNN using multi-step Leaky Integrate-and-Fire (LIF) neurons and attention mechanisms capable of processing spatio-temporal patterns over multiple time steps is designed. The architecture is further enhanced with state, action, and reward encodings to create a Transformer-like structure optimized for reinforcement le

Stats

• citations1
• S2 citations—
• github stars0
• HF likes0
• heat score2.26
• arxiv keyuddin2025energy

Related papers

• Deep Reinforcement Learning With Spiking Q-learning (2022)0.00
• Fully Spiking Actor Network With Intra-layer Connections For Reinforcement Learning (2024)0.00
• Scalable Multi-task Learning Through Spiking Neural Networks With Adaptive Task-switching Policy For Intelligent Autonomous Agents (2025)0.00
• Transformer Based Reinforcement Learning For Games (2019)0.00
• A Low Latency Adaptive Coding Spiking Framework For Deep Reinforcement Learning (2022)7.16
• Distilling Neuron Spike With High Temperature In Reinforcement Learning Agents (2021)0.00
• Human-level Control Through Directly-trained Deep Spiking Q-networks (2021)12.40
• Learning To Control Dynamical Agents Via Spiking Neural Networks And Metropolis-hastings Sampling (2025)0.00