Learning Zero-shot Cooperation With Humans, Assuming Humans Are Biased

Abstract

There is a recent trend of applying multi-agent reinforcement learning (MARL) to train an agent that can cooperate with humans in a zero-shot fashion without using any human data. The typical workflow is to first repeatedly run self-play (SP) to build a policy pool and then train the final adaptive policy against this pool. A crucial limitation of this framework is that every policy in the pool is optimized w.r.t. the environment reward function, which implicitly assumes that the testing partners of the adaptive policy will be precisely optimizing the same reward function as well. However, human objectives are often substantially biased according to their own preferences, which can differ greatly from the environment reward. We propose a more general framework, Hidden-Utility Self-Play (HSP), which explicitly models human biases as hidden reward functions in the self-play objective. By approximating the reward space as linear functions, HSP adopts an effective technique to generate an

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyyu2023learning

Related papers

• Collaborating With Humans Without Human Data (2021)0.00
• Role Play: Learning Adaptive Role-specific Strategies In Multi-agent Interactions (2024)0.00
• Maximum Entropy Population-based Training For Zero-shot Human-ai Coordination (2021)0.00
• Heterogeneous Multi-agent Reinforcement Learning For Zero-shot Scalable Collaboration (2024)6.34
• Towards Few-shot Coordination: Revisiting Ad-hoc Teamplay Challenge In The Game Of Hanabi (2023)0.00
• "other-play" For Zero-shot Coordination (2020)0.00
• Cooperative And Competitive Biases For Multi-agent Reinforcement Learning (2021)2.26
• Training Generalizable Collaborative Agents Via Strategic Risk Aversion (2026)0.00