Adapting Behaviour For Learning Progress

Abstract

Determining what experience to generate to best facilitate learning (i.e. exploration) is one of the distinguishing features and open challenges in reinforcement learning. The advent of distributed agents that interact with parallel instances of the environment has enabled larger scales and greater flexibility, but has not removed the need to tune exploration to the task, because the ideal data for the learning algorithm necessarily depends on its process of learning. We propose to dynamically adapt the data generation by using a non-stationary multi-armed bandit to optimize a proxy of the learning progress. The data distribution is controlled by modulating multiple parameters of the policy (such as stochasticity, consistency or optimism) without significant overhead. The adaptation speed of the bandit can be increased by exploiting the factored modulation structure. We demonstrate on a suite of Atari 2600 games how this unified approach produces results comparable to per-task tuning a

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyschaul2019adapting

Related papers

• A Bandit Framework For Optimal Selection Of Reinforcement Learning Agents (2019)0.00
• Dynamic Memory For Interpretable Sequential Optimisation (2022)0.00
• Adaptive Action Duration With Contextual Bandits For Deep Reinforcement Learning In Dynamic Environments (2025)0.00
• Principal-agent Bandit Games With Self-interested And Exploratory Learning Agents (2024)0.00
• Lifelong Reinforcement Learning Via Neuromodulation (2024)0.00
• Data Efficient Training For Reinforcement Learning With Adaptive Behavior Policy Sharing (2020)0.00
• Learning From Learners: Adapting Reinforcement Learning Agents To Be Competitive In A Card Game (2020)0.00
• Policy Learning With Adaptively Collected Data (2021)0.00