Human-level Reinforcement Learning Through Theory-based Modeling, Exploration, And Planning

Abstract

Reinforcement learning (RL) studies how an agent comes to achieve reward in an environment through interactions over time. Recent advances in machine RL have surpassed human expertise at the world's oldest board games and many classic video games, but they require vast quantities of experience to learn successfully -- none of today's algorithms account for the human ability to learn so many different tasks, so quickly. Here we propose a new approach to this challenge based on a particularly strong form of model-based RL which we call Theory-Based Reinforcement Learning, because it uses human-like intuitive theories -- rich, abstract, causal models of physical objects, intentional agents, and their interactions -- to explore and model an environment, and plan effectively to achieve task goals. We instantiate the approach in a video game playing agent called EMPA (the Exploring, Modeling, and Planning Agent), which performs Bayesian inference to learn probabilistic generative models expr

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keytsividis2021human

Related papers

• Synthesizing World Models For Bilevel Planning (2025)0.00
• Fast Exploration With Simplified Models And Approximately Optimistic Planning In Model Based Reinforcement Learning (2018)0.00
• Mapping Out The Space Of Human Feedback For Reinforcement Learning: A Conceptual Framework (2024)0.00
• Multi-agent Reinforcement Learning: A Selective Overview Of Theories And Algorithms (2019)21.85
• A Human Mixed Strategy Approach To Deep Reinforcement Learning (2018)7.50
• Modeling Human Exploration Through Resource-rational Reinforcement Learning (2022)2.26
• Comparing Reinforcement Learning And Human Learning Using The Game Of Hidden Rules (2023)2.26
• Learning Human Rewards By Inferring Their Latent Intelligence Levels In Multi-agent Games: A Theory-of-mind Approach With Application To Driving Data (2021)0.00