Partial Models For Building Adaptive Model-based Reinforcement Learning Agents

Abstract

In neuroscience, one of the key behavioral tests for determining whether a subject of study exhibits model-based behavior is to study its adaptiveness to local changes in the environment. In reinforcement learning, however, recent studies have shown that modern model-based agents display poor adaptivity to such changes. The main reason for this is that modern agents are typically designed to improve sample efficiency in single task settings and thus do not take into account the challenges that can arise in other settings. In local adaptation settings, one particularly important challenge is in quickly building and maintaining a sufficiently accurate model after a local change. This is challenging for deep model-based agents as their models and replay buffers are monolithic structures lacking distribution shift handling capabilities. In this study, we show that the conceptually simple idea of partial models can allow deep model-based agents to overcome this challenge and thus allow for

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyalver2024partial

Related papers

• Replay Buffer With Local Forgetting For Adapting To Local Environment Changes In Deep Model-based Reinforcement Learning (2023)0.00
• Towards Evaluating Adaptivity Of Model-based Reinforcement Learning Methods (2022)0.00
• Models As Agents: Optimizing Multi-step Predictions Of Interactive Local Models In Model-based Multi-agent Reinforcement Learning (2023)6.77
• Self-adapting Goals Allow Transfer Of Predictive Models To New Tasks (2019)2.26
• Discovering Latent States For Model Learning: Applying Sensorimotor Contingencies Theory And Predictive Processing To Model Context (2016)0.00
• Reinforcement Learning With Brain-inspired Modulation Can Improve Adaptation To Environmental Changes (2022)0.00
• Agent Modelling Under Partial Observability For Deep Reinforcement Learning (2020)0.00
• Deep Online Learning Via Meta-learning: Continual Adaptation For Model-based RL (2018)0.00