Reinforcement Learning And Inverse Reinforcement Learning With System 1 And System 2

Abstract

Inferring a person's goal from their behavior is an important problem in applications of AI (e.g. automated assistants, recommender systems). The workhorse model for this task is the rational actor model - this amounts to assuming that people have stable reward functions, discount the future exponentially, and construct optimal plans. Under the rational actor assumption techniques such as inverse reinforcement learning (IRL) can be used to infer a person's goals from their actions. A competing model is the dual-system model. Here decisions are the result of an interplay between a fast, automatic, heuristic-based system 1 and a slower, deliberate, calculating system 2. We generalize the dual system framework to the case of Markov decision problems and show how to compute optimal plans for dual-system agents. We show that dual-system agents exhibit behaviors that are incompatible with rational actor assumption. We show that naive applications of rational-actor IRL to the behavior of dual

Stats

• citations8
• S2 citations—
• github stars0
• HF likes0
• heat score7.16
• arxiv keypeysakhovich2018reinforcement

Related papers

• Inverse Reinforcement Learning With Simultaneous Estimation Of Rewards And Dynamics (2016)0.00
• Basis For Intentions: Efficient Inverse Reinforcement Learning Using Past Experience (2022)0.00
• Modeling And Interpreting Real-world Human Risk Decision Making With Inverse Reinforcement Learning (2019)0.00
• Accounting For Human Learning When Inferring Human Preferences (2020)0.00
• Learning Human Rewards By Inferring Their Latent Intelligence Levels In Multi-agent Games: A Theory-of-mind Approach With Application To Driving Data (2021)0.00
• Cooperative Inverse Reinforcement Learning (2016)0.00
• Where Do You Think You're Going?: Inferring Beliefs About Dynamics From Behavior (2018)0.00
• Multi-agent Inverse Reinforcement Learning: Suboptimal Demonstrations And Alternative Solution Concepts (2021)0.00