Towards Human-like RL: Taming Non-naturalistic Behavior In Deep RL Via Adaptive Behavioral Costs In 3D Games

Abstract

In this paper, we propose a new approach called Adaptive Behavioral Costs in Reinforcement Learning (ABC-RL) for training a human-like agent with competitive strength. While deep reinforcement learning agents have recently achieved superhuman performance in various video games, some of these unconstrained agents may exhibit actions, such as shaking and spinning, that are not typically observed in human behavior, resulting in peculiar gameplay experiences. To behave like humans and retain similar performance, ABC-RL augments behavioral limitations as cost signals in reinforcement learning with dynamically adjusted weights. Unlike traditional constrained policy optimization, we propose a new formulation that minimizes the behavioral costs subject to a constraint of the value function. By leveraging the augmented Lagrangian, our approach is an approximation of the Lagrangian adjustment, which handles the trade-off between the performance and the human-like behavior. Through experiments co

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyho2023towards

Related papers

• A Human Mixed Strategy Approach To Deep Reinforcement Learning (2018)7.50
• Data Efficient Training For Reinforcement Learning With Adaptive Behavior Policy Sharing (2020)0.00
• BCR-DRL: Behavior- And Context-aware Reward For Deep Reinforcement Learning In Human-ai Coordination (2024)0.00
• Learnable Behavior Control: Breaking Atari Human World Records Via Sample-efficient Behavior Selection (2023)0.00
• Human-level Reinforcement Learning Through Theory-based Modeling, Exploration, And Planning (2021)0.00
• A Low-cost Ethics Shaping Approach For Designing Reinforcement Learning Agents (2017)0.00
• Adversarial Policies: Attacking Deep Reinforcement Learning (2019)0.00
• Human-inspired Framework To Accelerate Reinforcement Learning (2023)0.00