Combining Imitation And Deep Reinforcement Learning To Accomplish Human-level Performance On A Virtual Foraging Task

Abstract

We develop a simple framework to learn bio-inspired foraging policies using human data. We conduct an experiment where humans are virtually immersed in an open field foraging environment and are trained to collect the highest amount of rewards. A Markov Decision Process (MDP) framework is introduced to model the human decision dynamics. Then, Imitation Learning (IL) based on maximum likelihood estimation is used to train Neural Networks (NN) that map human decisions to observed states. The results show that passive imitation substantially underperforms humans. We further refine the human-inspired policies via Reinforcement Learning (RL) using the on-policy Proximal Policy Optimization (PPO) algorithm which shows better stability than other algorithms and can steadily improve the policies pretrained with IL. We show that the combination of IL and RL can match human results and that good performance strongly depends on combining the allocentric information with an egocentric representati

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keygiammarino2022combining

Related papers

• Adaptive Patch Foraging In Deep Reinforcement Learning Agents (2022)0.00
• A New Framework For Query Efficient Active Imitation Learning (2019)0.00
• Explore, Exploit Or Listen: Combining Human Feedback And Policy Model To Speed Up Deep Reinforcement Learning In 3D Worlds (2017)0.00
• Inverse Rational Control: Inferring What You Think From How You Forage (2018)0.00
• Modeling Human Exploration Through Resource-rational Reinforcement Learning (2022)2.26
• General Policy Mapping: Online Continual Reinforcement Learning Inspired On The Insect Brain (2022)0.00
• Efficient Deep Reinforcement Learning With Predictive Processing Proximal Policy Optimization (2022)0.00
• Improving Multimodal Interactive Agents With Reinforcement Learning From Human Feedback (2022)0.00