Scalable Multi-agent Inverse Reinforcement Learning Via Actor-attention-critic

Abstract

Multi-agent adversarial inverse reinforcement learning (MA-AIRL) is a recent approach that applies single-agent AIRL to multi-agent problems where we seek to recover both policies for our agents and reward functions that promote expert-like behavior. While MA-AIRL has promising results on cooperative and competitive tasks, it is sample-inefficient and has only been validated empirically for small numbers of agents -- its ability to scale to many agents remains an open question. We propose a multi-agent inverse RL algorithm that is more sample-efficient and scalable than previous works. Specifically, we employ multi-agent actor-attention-critic (MAAC) -- an off-policy multi-agent RL (MARL) method -- for the RL inner loop of the inverse RL procedure. In doing so, we are able to increase sample efficiency compared to state-of-the-art baselines, across both small- and large-scale tasks. Moreover, the RL agents trained on the rewards recovered by our method better match the experts than tho

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyjeon2020scalable

Related papers

• Actor-attention-critic For Multi-agent Reinforcement Learning (2018)0.00
• On Feasible Rewards In Multi-agent Inverse Reinforcement Learning (2024)0.00
• Rethinking Adversarial Inverse Reinforcement Learning: Policy Imitation, Transferable Reward Recovery And Algebraic Equilibrium Proof (2024)0.00
• Multi-agent Reinforcement Learning Accelerated MCMC On Multiscale Inversion Problem (2020)0.00
• Attention Actor-critic Algorithm For Multi-agent Constrained Co-operative Reinforcement Learning (2021)0.00
• Learning Robust Rewards With Adversarial Inverse Reinforcement Learning (2017)0.00
• AIR: Unifying Individual And Collective Exploration In Cooperative Multi-agent Reinforcement Learning (2024)0.00
• Efficient Distributed Framework For Collaborative Multi-agent Reinforcement Learning (2022)0.00