Differentiable Architecture Search For Reinforcement Learning

Abstract

In this paper, we investigate the fundamental question: To what extent are gradient-based neural architecture search (NAS) techniques applicable to RL? Using the original DARTS as a convenient baseline, we discover that the discrete architectures found can achieve up to 250% performance compared to manual architecture designs on both discrete and continuous action space environments across off-policy and on-policy RL algorithms, at only 3x more computation time. Furthermore, through numerous ablation studies, we systematically verify that not only does DARTS correctly upweight operations during its supernet phrase, but also gradually improves resulting discrete cells up to 30x more efficiently than random search, suggesting DARTS is surprisingly an effective tool for improving architectures in RL.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keymiao2021differentiable

Related papers

• Optimizing The Neural Architecture Of Reinforcement Learning Agents (2020)0.00
• An Integrated Approach To Neural Architecture Search For Deep Q-networks (2025)0.00
• Policy Gradient RL Algorithms As Directed Acyclic Graphs (2020)0.00
• Adapting The Function Approximation Architecture In Online Reinforcement Learning (2021)0.00
• Efficient Multi-turn RL For GUI Agents Via Decoupled Training And Adaptive Data Curation (2025)0.00
• A3C-S: Automated Agent Accelerator Co-search Towards Efficient Deep Reinforcement Learning (2021)0.00
• Delayed Geometric Discounts: An Alternative Criterion For Reinforcement Learning (2022)0.00
• Neural Architecture Evolution In Deep Reinforcement Learning For Continuous Control (2019)0.00