ELIGN: Expectation Alignment As A Multi-agent Intrinsic Reward

Abstract

Modern multi-agent reinforcement learning frameworks rely on centralized training and reward shaping to perform well. However, centralized training and dense rewards are not readily available in the real world. Current multi-agent algorithms struggle to learn in the alternative setup of decentralized training or sparse rewards. To address these issues, we propose a self-supervised intrinsic reward ELIGN - expectation alignment - inspired by the self-organization principle in Zoology. Similar to how animals collaborate in a decentralized manner with those in their vicinity, agents trained with expectation alignment learn behaviors that match their neighbors' expectations. This allows the agents to learn collaborative behaviors without any external reward or centralized training. We demonstrate the efficacy of our approach across 6 tasks in the multi-agent particle and the complex Google Research football environments, comparing ELIGN to sparse and curiosity-based intrinsic rewards. When

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyma2022elign

Related papers

• Coordinated Exploration Via Intrinsic Rewards For Multi-agent Reinforcement Learning (2019)0.00
• Exploration With Unreliable Intrinsic Reward In Multi-agent Reinforcement Learning (2019)0.00
• Individual Contributions As Intrinsic Exploration Scaffolds For Multi-agent Reinforcement Learning (2024)2.80
• Scalable Agent Alignment Via Reward Modeling: A Research Direction (2018)0.00
• From Centralized To Self-supervised: Pursuing Realistic Multi-agent Reinforcement Learning (2023)0.00
• LIGS: Learnable Intrinsic-reward Generation Selection For Multi-agent Learning (2021)0.00
• Emotion-inspired Learning Signals (EILS): A Homeostatic Framework For Adaptive Autonomous Agents (2025)0.00
• Discovering Individual Rewards In Collective Behavior Through Inverse Multi-agent Reinforcement Learning (2023)0.00