Eau De \(q\)-network: Adaptive Distillation Of Neural Networks In Deep Reinforcement Learning

Abstract

Recent works have successfully demonstrated that sparse deep reinforcement learning agents can be competitive against their dense counterparts. This opens up opportunities for reinforcement learning applications in fields where inference time and memory requirements are cost-sensitive or limited by hardware. Until now, dense-to-sparse methods have relied on hand-designed sparsity schedules that are not synchronized with the agent's learning pace. Crucially, the final sparsity level is chosen as a hyperparameter, which requires careful tuning as setting it too high might lead to poor performances. In this work, we address these shortcomings by crafting a dense-to-sparse algorithm that we name Eau De \(Q\)-Network (EauDeQN). To increase sparsity at the agent's learning pace, we consider multiple online networks with different sparsity levels, where each online network is trained from a shared target network. At each target update, the online network with the smallest loss is chosen as th

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyvincent2025eau

Related papers

• Adaptive \(q\)-network: On-the-fly Target Selection For Deep Reinforcement Learning (2024)0.00
• Dynamic Sparse Training For Deep Reinforcement Learning (2021)0.00
• Automaton Distillation: Neuro-symbolic Transfer Learning For Deep Reinforcement Learning (2023)0.00
• DQN With Model-based Exploration: Efficient Learning On Environments With Sparse Rewards (2019)0.00
• Deep Reinforcement Learning With Spiking Q-learning (2022)0.00
• An Adaptive Synchronization Approach For Weights Of Deep Reinforcement Learning (2020)0.00
• Learning Sparse Representations Incrementally In Deep Reinforcement Learning (2019)0.00
• Sampling Efficient Deep Reinforcement Learning Through Preference-guided Stochastic Exploration (2022)8.09