Metacontrol For Adaptive Imagination-based Optimization

Abstract

Many machine learning systems are built to solve the hardest examples of a particular task, which often makes them large and expensive to run---especially with respect to the easier examples, which might require much less computation. For an agent with a limited computational budget, this "one-size-fits-all" approach may result in the agent wasting valuable computation on easy examples, while not spending enough on hard examples. Rather than learning a single, fixed policy for solving all instances of a task, we introduce a metacontroller which learns to optimize a sequence of "imagined" internal simulations over predictive models of the world in order to construct a more informed, and more economical, solution. The metacontroller component is a model-free reinforcement learning agent, which decides both how many iterations of the optimization procedure to run, as well as which model to consult on each iteration. The models (which we call "experts") can be state transition models, acti

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyhamrick2017metacontrol

Related papers

• Towards An Adaptable And Generalizable Optimization Engine In Decision And Control: A Meta Reinforcement Learning Approach (2024)0.00
• Meta-trained Agents Implement Bayes-optimal Agents (2020)0.00
• Meta-learning Within Projective Simulation (2016)10.85
• Partial Models For Building Adaptive Model-based Reinforcement Learning Agents (2024)0.00
• Dream To Explore: Adaptive Simulations For Autonomous Systems (2021)0.00
• Finding Useful Predictions By Meta-gradient Descent To Improve Decision-making (2021)0.00
• Context Meta-reinforcement Learning Via Neuromodulation (2021)6.34
• Efficient Meta Reinforcement Learning For Preference-based Fast Adaptation (2022)0.00